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Author’s Preface to Volumes | and II 


Random processes are of great and ever increasing interest to 
scientists and engineers working in vatious branches of radio 
physics. This is quite understandable, since at the present stage 
of technological development, random noise constitutes the chief 
obstacle to further improvement of certain engineering devices. 
In fact, a current problem in many fields of radio physics is that of 
evolving techniques for analysis and design of equipment which 
is immune to noise. Convincing proof of the importance and 
urgency of this subject is afforded by the large number of books 
devoted to it, published both in the U.S.S.R. and elsewhere, 

The present book, in two volumes, is concerned with a variety 
of topics which have been left to one side, as it were, but which are 
nevertheless of great practical importance. In particular, we have 
in mind problems involving the effects of noise on electronic 
relays and vacuum-tube oscillators. Recently, there has been 
increased interest in the operation of switching devices and self- 
excited systems, not only because of the development of methods 
of coherent radar detection, but also in connection with such 
problems as increasing the accuracy of time measurements, 
designing particle accelerators and computing machines, etc. 
However, the journal literature in these new fields of statistical 
radio engineering is hard to penetrate. In the present book, noise 
in relays and oscillators is studied in an organized fashion, from a 
rather general point of view. Material that can be found in the 
literature has been systematized, and the results of some original 
investigations by the author have been included. The physical 
treatment of the results obtained may be a bit sketchy, but it 
‘seems to us that brevity coupled with wide coverage is justified 


in this instance, since the rapid growth in the total number of 


§ v 


vi AUTHOR'S PREFACE 


published papers has increased the need for some kind of survey, 
even one resembling a handbook. 

From a mathematical point of view, the topics studied here 
involve nonlinear transformations of random functions, mostly 
transformations with memory. Certain mathematical methods are 
worked out in a form suitable for application to problems of radio 
engineering, and are then exploited systematically. As the reader 
will discover, the following methods turn out to be effective: 

1, Reduction of a given process to a process without aftereffect, 
followed by application of Markov process techniques, in parti- 
cular, the Fokker-Planck equation; 

Linearization of the original equations, which allows us to apply 
correlation theory in the linear approximation; 


ern 


The quasi-static approach, which allows us to reduce a non- 
linear transformation with memory to one without memory. 


\ The treatments in the available literature usually deal only with 
the correlation theory, or only with the theory of Markov processes, 
However, to solve concrete problems, one must be able to go from 
the methods of either theory to those of the other, choosing the 
best methods for solving the problem at hand. For this reason, 
we pay special attention to the problem of when, and in what 
sense, the theory of Markov processes can be applied to actual 
fluctuations, which are originally described in terms of the correla- 
tion theory. 

Part | of Volume I is devoted to a review of the mathematical 
results used later in the book, In many places, it has seemed 
appropriate, for the sake of brevity, to depart from the usual 
presentation of the basic concepts and results of probability 
theory. The reader who is not familiar with this necessary back- 
ground material can refer to any one of a number of textbooks, 
Those already acquainted with elementary probability theory will 
find that Part | contains a different treatment of certain well-known 
topics, in addition to some new results. In fact, Part | ought not to 
be devoid of interest even to mathematicians, provided, of course, 
that they make due allowances for the heuristic level of rigor adopted. 
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In Volume I, Part 2 and in Volume II, we systematically use the 
methods just enumerated to solve specific problems of statistical 
radio engineering. At the same time, we try to explain the conditions 
under which the various methods of analyzing noise phenomena 
are applicable, Perhaps some of the methods are given in more 
detail than is necessary for immediate applications, but this is done 
with a view to further development of the tools needed for future 
work, It is reasonable to expect that the progressively more compli- 
cated problems which come up in engineering practice will require 
the use of progressively more sophisticated methods of analysis. 
The author will regard his task as accomplished if this book 
serves in some measure to broaden the theoretical outlook of 
radio scientists, by helping them master certain methods which 
they may find novel at first. Although the material given here is 
intended primarily for specialists in the field of statistical radio 
engineering, it should also interest scientists working in other 
fields where similar statistical methods can be profitably used. 

As a tule, no attempt is made to cite the papers where various 
results appear for the first time. This is because any attempt to 
make systematic source references would be likely to entail errors, 
unless one wishes to become involved in special bibliographic 
research. 

Finally, the author would like to take this opportunity to thank 
V. 1, Tikhonov, who suggested that the present book be written 
and rendered substantial assistance, Y. L. Klimontovich, who in 
his capacity as editor made many valuable suggestions while the 
manuscript was being prepared for the printer, and I. G. Akopyan, 
who constructed the figures appearing in Volume II, Chapter 9, 
The author would also like to thank (among others) S. P. Strelkov, 
S. D. Gvozdover, B. R. Levin, S, A. Akhmanov, Y. M. Romanovski 
and P, S. Lande, for their help with the manuscript. He also 
expresses his gratitude to Dr. R. A. Silverman for undertaking 
to translate and edit the English-language edition of this book, 
thereby greatly enlarging its prospective audience. 


R.L.S. 


Translator’s Preface to Volume | 


_ The present book is the first of two volumes dealing with 
various topics in the theory of random noise. This material is for 
the most part not available elsewhere in the English-language 
literature, and is in fact mainly due to Dr. Stratonovich and his 
co-workers. Speaking as one who has long been interested in 
noise theory, I feel confident that a large audience of scientists and 
engineers will find the contents of this volume interesting, timely, 
and most important of all, highly original. 

_ I have worked through all the mathematical derivations in the 
book, and in the process have detected and corrected some typo- 
graphical errors which had survived in the Russian edition. Various 
small improvements that occurred to me as I studied the book have 
been incorporated without special comment. In reply to my 
{queries, Dr. Stratonovich has cleared up a few stubborn points 
that I was unable to resolve by myself, and the present edition 
‘benefits from these clarifications. For example, the supplement to 
‘Chapter 4, giving the details of a rather tricky argument, stems 
from a letter by the author. 
} Passages appearing between boldface asterisks were printed in 
ibrevier originally, They relate to certain side issues which can be 
Somitted without loss of continuity. Finally, bearing in mind the 
‘needs of its readers, I have suitably modified the list of books cited 
tin the Bibliography. 
i RAS. 
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PART 1 


General Theory of 
Random Processes 


CHAPTER 1 


Random Functions and Their 
Statistical Characteristics 


1. Random Variables 


Roughly speaking, a random variable € is characterized by the 
fact that instead of knowing its precise value, we only know how to 
obtain various values of £ under certain experimental conditions 
which are fixed, stable and physically well-defined. As a result of 
urknown factors, in a given experiment £ takes a particular value 
&,, which we call a realization (or sample value) of the random 
variable ¢. For example, consider the number of points which 
appear when a die is tossed, or the reading of a meter in a certain 
apparatus at a given instant of time. By repeating the experiment 
many times (and this possibility is implicit in the definition of a 
random variable), we can always obtain a large number of realiza- 
tions. Therefore, a random variable can also be characterized by a 
certain statistical ensemble of realizations, By subjecting the realiza- 
‘tions of a given random variable to statistical data processing, we 
can find certain of its statistical characteristics, Therefore, when we 
say that a random variable is “given,” we mean that we know the 
statistical data which completely characterize it. 

The simplest statistical characteristic of a random variable is its 
‘mban value: If €,, €,,... are realizations of the random variable ¢, 
then the mean value of é can be defined by 


a 


bie ep tbe ton + be 
; CE) = lim #11 38 oT Ee, 


(1-1) 


3 


4 GENERAL THEORY OF RANDOM PROCESSES [cu. 1 


ice., <£> is the limit of the arithmetic mean of the sample values as 
the number of sample values (with respect to which the averaging is 
performed) is increased without limit. Probability theory can only 
be used to study experimental data for which such limits exist and 
do not depend on how the realizations £,, £,, ..., &, are chosen from 
the total statistical ensemble. Naturally, in making this choice, no 
information involving the specific values of the realizations should 
be used.! 

Other statistical characteristics of a random variable £ can be 
defined by first going over to a new random variable y = f(€), and 
then using formula (1.1) to calculate <n); in so doing, we explicitly 
use the form of the function f(é). Thus, setting f(f) = £2, we 
define the mean square of & by the formula 


G+ Gt te 


n 


<> = lim (1.2) 


As another example, consider the probability P{é << x} that the 
inequality € << x holds, where x is a fixed number. To calculate 
P{é < x}, we introduce the random variable 
1 for z>0, 
« = Hx — &), where zs) = 0 for <0. (1.3) 
Then we have 
PEE < x} = He — &), 








1 We address the following remark to the reader who is already acquainted 
with the subject: The approach based on the limit (1.1) is simple, intuitive 
and quite adequate for use in the applications; it has certain defects from a 
formal mathematical point of view, but these are unimportant when dealing 
with applications of probability theory. The point is that as far as the 
applications are concerned, the assertion that a given event has probability 
equal to unity (or arbitrarily close to unity) is equivalent to the assertion 
that the event occurs. Therefore, instead of the familiar requirement that 
the convergence expressed by (1.1) occurs with probability arbitrarily 
close to unity, we choose as our starting point the requirement that (1.1) 
actually converges. The reader will find other ways of dealing with this 
point in other treatments of the subject. 
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ke, 

P(E <x) = lim SEE Fe (1.4) 

nao n 

In fact, bearing in mind the definition of the function #(z), we see 
that the average of the arithmetic mean (e; + €2 + ... + €,)/m is 
just the ratio mjn, where m is the number of inequalities 
&,< x (1 <i <n) which are valid, and n is the total number of 
trials. Thus, (1.4) can be written in the familiar form 


Pte <x} =lim™, (1.5) 
jim 


The mean value <9 (x — €)> regarded as a function of x, ie., 


F(x) = x — £)), (1.6) 


tis called the distribution function of £, and its derivative 
: 4 : 
ude) = FAs) (FQ) =f ade) ae’) (1.7) 


is called the probability density of £. Substituting (1.6) into (1.7), and 
formally differentiating under the averaging sign, we obtain 


wx) = (KE — x)>, (1.8) 


where 
He) = EH 
) = FX) 
is the Dirac delta function, which has the property that 


fe f(%) 8(z — 29) dz = f(z) for any function f(z) . 


In what follows, we shall often be concerned with probability 
densities. For brevity, it is sometimes convenient to denote the 
value of x by the same letter as the random variable ¢ itself, so 
that w,(£) becomes w,(£). We then drop the redundant subscript ¢ 
and write simply w(¢) for the probability density of the random 
variable £, 
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The probability density w(£) satisfies the normalization condition® 


fae =i, (1.9) 
which, according to (1.8), is equivalent to the trivial identity 


db =1. 


A random variable £ is completely characterized by its probability 
density, and knowing this probability density, we can calculate the 
mean value </(é)> of any function f(£). In fact, using the basic 
property of the delta function, we can first write f(€) as 


HE) = [ fle) BE — ») dx 


and then calculate ¢ /(€)>. Taking the average under the integration 
sign, we see at once that 
<ME)> = f fe) <8(E — =)) de, 


or 
MED = J f(s) we) de, (1.10) 


because of (1.8). 

Next, we show how the probability density behaves under 
transformations of the original random variable €. Let the new 
random variable 7 be defined by 


n = af), (LLL) 


where g(x) is a known function. Applying (1.8), we write the 
required probability density of the new random variable y in the form 


w,(y) = <8(n — y)> = <d[a(é) — yD- 


? When limits of integration are omitted, they will always be understood 
to be from —o to, 


' 
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J , 
It then follows from (1.10) that 


w,(y) = f Slee) — yl edz) dx, (1.12) 


or more concisely, 
\ 


ww(n) = f Blg(e) — ] we) dé. (1.13) 


This last integral can be evaluated by using properties of the delta 
function. If the function (1.11) is monotone, it has a unique 
inverse function 


E=hn) (h=g"). (1.14) 
In this case, it follows froin (1.12) that 
; __ wf _ dt 
, wo(n) = Ese = oth) | Fence (1.15) 
7 dE Nunta) 





i 
In the general case where the function g(£) is not monotone, and 
instead has “several inverse functions’ 


f=hty) G=1L2.07, (1.16) 


formula (1.12) gives 
-t 


=> £ 
a ~n) = Zxthan) | aa ewe (1.17) 
f 


| Besides the probability density w(£), the characteristic function 
Osu) = <emy = f etmta(e) de (1.18) 


completely characterizes the random variable €. In fact, w(€) is 
just the Fourier transform of @,(u): 


w(é) = Ao f edu) du (1.19) 


8 GENERAL THEORY OF RANDOM PROCESSES [cH. 1 


The moments 
ty, = E") (nt = 1,2, ) (1.20) 


of the random variable € can be obtained from the-characteristic 
function by differentiation 
_ 1d Ou) 


Ma = a — ‘ 
a i du® lao 


(1.21) 
From a knowledge of the moments m,, i.e., the values of the 
derivatives of the characteristic function @,(u) at the origin, we 
can write @,(u) as a Maclaurin series: 





= (5,)\0 
Ow = 14+ % a (1.22) 
Of course, we can only write (1.22) when the moments m, are 
finite and do not grow too rapidly as the index 7 is increased. 

To completely describe a random variable, one has to know all 
its Moments m,, my, .... but most of the physically important 
information is contained in the first few moments. The higher 
moments describe rather unimportant details in the behavior of the 
probability density, which pertain mostly to extreme values of the 
random variable. For a variety of reasons, it is more convenient to 
characterize a random variable not by its moments, but by its 
cumulants (ot semi-invariants) k,, defined by the relation 





ni 


xe) = exp { uy" hal (1.23) 


Comparing (1.22) and (1.23), we can obtain the following formulas 
relating the moments and the cumulants: 


ky = my — 3mm, + 2m, (1.24) 


ky = m, — 3mz — 4mm, + 12mim, — 6mi, 
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from the structure of these formulas, it is clear that if we know the 
tirst n moments, we can calculate the first 2 cumulants, and con- 
versely, i,e., the first 2 cumulants contain just as much information 
as the first 2 moments. If we confine ourselves to a definite number 
of first cumulants, it is convenient to set the higher cumulants equal 
to zero. To deal with moments in the same way, i.e., to set all 
moments higher than a certain order equal to zero, is not desirable, 
since then the characteristic function would behave at infinity in a 
way which makes it difficult to calculate the inversion integral (1.19). 
In fact, w(€) would then become a sum of derivatives of delta 
functions, with no simple intuitive meaning. 

The first cumulant is the same as the first moment, i.e., 
m, = k, = (>, and is just the mean value, the most important 
characteristic of a random variable. The next two cumulants are 
the central moments of orders two and three: 


ke=(E—<OY), he = CE-CO!- (1.25) 
) 
Fhe quantity k, = <7) — ¢€* is called the variance (or dis- 
persion) of £, and is denoted by 
r 


Dé = ¢) — <f. (1.26) 


The variance is also a very important characteristic of a random 
variable, since the smaller the variance, the more exact our 
knowledge of the random variable, The probability that the absolute 
value of the deviation of £ from its mean value <£> exceeds a fixed 
«> 0 approaches zero as Dé approaches zero, Therefore, if Dé 
is small enough, we can assume that € differs by very little from ¢£>. 
- Instead of the variance, one often deals with the standard 
deviation 


o(£) = [Dep = [cé> — C7]. (1.27) 


While retaining the important features of the variance, the standard 
deviation has the advantage that it has the same physical dimension 
as the random variable itself. The standard deviation indicates the 
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order of magnitude of random deviations from the mean value, i.e., 

& — <& ~ of€). (1.28) 

For the time being, we shall not discuss the significance of higher 
cumulants. However, we note that the ratios 


k k, 
Y= Bi = 5 (1.29) 


are called the coefficient of asymmetry and the coefficient of excess, 
respectively. The physical meaning of the cumulant &, decreases as 
n increases. 


2. Correlation Between Random Variables 


Suppose we have r random variables ,, ..., ,. Loosely speaking, 
this means that as a result of each experiment (trial) we simul- 
taneously obtain a realization of each of the r random variables. 
Vhe random variables &,,...,&, are completely characterized by 
the (jomt) r-dimensional probability density 


Weyerre (May soy Xp) = (8(E, — xy). HE, — 2), (1.30) 


or simply w(&,, ..., €,), which is a function of r variables. It follows 
by definition that the mean value of any function /(é,, ..., €,) can 
be calculated by integrating with weight w(é,,..., €,): 


(Albino EDD = fom [ Mbay os 62) Ein vn Ge) dm be (131) 

If we set f = 1, we obtain the normalization condition 
J J bn nn Bn) dl dp = 1 (1.32) 
Next, we divide the r random variables ¢,, ..., §, into two groups 


4, 5 & and €,43, 5 &,, and we ask whether knowing the actual 
realizations of the random variables ¢,,,,..., € gives us any 
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information about &,,..., €,. For example, suppose we have two 
random variables £, and £,, described by a probability density 
w(é,, §2). In the absence of knowledge of the actual realization 
(i.e., pbserved value) of £, the random variable £, has the prob- 
atality density 

a (Ei) = f my &) dee (1.33) 

+ 

Knowing the observed value of the random variable £, (which gives 
us complete information about £,), we can also obtain some infor- 
mation about the other random variable £,. Of course, in general 
our khowledge of €, is still imperfect, even when we know the 
observed value of &,, and hence £, remains undetermined, i.e., is 
still alrandom variable. However, €, is now described by another 
probability density 


— _ Es &) = w(E,, £2) 34 
w( | £2) Tole, &) db, é,) dé, we)” (1.34) 


which contains more (or at least not less) information about ¢, 
thari the distribution (1.33).2 The expression (1.34) is called the 
conditional probability density of £,, given the value of &. The 
integral appearing in the denominator of the middle term in (1.34) 
guarantees that the normalization condition 


‘ 


flr | &e) df = 1 (135) 


Y 
is satisfied. 

Just how much information is gained about £, by knowing the 
value of £, depends on the particular case. In some cases, no 
information about €, is gained, regardless of the observed value of 
2, which means that 


1 w(Es | €2) = w(h) - (1.36) 


* Asis stomary, probability densities will often be referred to (somewhat 
imprecisely) as ‘‘distributions,” 
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We see from (1.34) that in this case w(£;, €,) is just the product of 
the two one-dimensional probability densities w(¢,) and w(é,), i.e., 


(Ex, £2) = w(E,) (£2) » (1.37) 


and the random variables £, and &, are said to be (statistically) 
independent. The opposite extreme is the case where knowledge of 
€, completely determines the other random variable £,, which is a 
function of £, [ = f(é,)]. Then, ¢, and €, are said to be completely 
correlated, and their joint distribution contains a delta function: 


w(Er, 2) = SE, — f(E2)] wl Ee) - (1.38) 


Naturally, there is a whole range of intermediate cases between 
these two extreme cases. 

All that has been said above about two random variables can 
immediately be generalized to the case of several random variables. 
Given r random variables £,, ..., £,, knowledge of several of them, 
€.g., Exits ++ &, increases (or at least does not decrease) our infor- 
mation about the other random variables &,, ..., &,. We can express 
the conditional probability density of €,,...,£, in terms of the 
original probability density w(£,, ..., £,) as follows: 


wo( Gas oo Eu Exe op) = wee, (139) 
If knowledge of £,,1, ..., £, does not increase our information about 
&,, « & then the two groups of random variables &;, ..., £, and 
Sepp oy €, are said to be (statistically) independent. However, if the 
conditional distribution (1.39) does not coincide with the “uncon- 
ditional” distribution 


wtb on) = fm fellas om Ed dbaen dé, MO) 


then there is statistical dependence between the two groups of 
random variables. 

It is now convenient to introduce numerical characteristics which 
describe the degree of statistical dependence between random 
variables, and which vanish when there is no statistical dependence 


Sec. 2] RANDOM FUNCTIONS AND THEIR CHARACTERISTICS 13 


¢ween them. Such characteristics are provided by multiple 
cotrelations of various orders. The covariance (synonymously, cross 
correlation or double correlation) of two random variables ¢, and £,, 
denoted by K[é,, &,], is defined by the relation 


K(f, $e] = (fib — (E> (fa - (1.41) 


If £, and £, are independent, we have 


(ff) = CED <b>» 


according to (1.37). Hence, in this case, the covariance vanishes. 
By definition, the covariance of a random variable with itself is just 


its variance 
® K(é, (] = Dé. (1-42) 


Sometimes, instead of K[é,, €], it is convenient to consider the 
dimensionless quantity 


, _ p__Klév Gil 

Ree R= Ey ole)’ ie 
which is proportional to K[é,, £,]. The quantity (1.43) reduces to 
uhity when the random variables coincide, and is called the 
cbrrelation coefficient of £, and £,. 

‘ Besides double correlations, there are multiple correlations of 
higher orders. Thus, given three random variables £,, &, & 
we can construct a triple correlation K[£,, £2, £5], which describes 
statistical dependence involving all three random variables £,, £2, £5, 
and vanishes when at least one of the random variables is inde- 
pendent of the others. We first form the quantity 
1 


Eibats> — <> (Ea (E> » (1.44) 


by subtracting the product of the mean values of the random 
variables from their joint third moment. The quantity (1.44) has 
the property that it vanishes when all three random variablés are 
independent of each other; however, (1.44) can be different from 
zero not only because of correlations involving all three random 
variables, but also because of correlations between pairs of random 
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variables. For example, suppose that one of the random variables, 
€; say, is independent of the other two. Then, there is no triple 
correlation, but the quantity (1.44) can be different from zero 
because of the presence of double correlation between £, and £,. 
In fact, in this case, we have (£,£¢,> = <&£2> (f>, and (1.44) 


equals 
(és) KE, &2] - (1.45) 


Thus, it is clear that to properly define the triple correlation 
K[é,, &, £5], we have to subtract from (1.44) all terms of the form 
(1.45) due to the presence of double correlations. There are three 
distinct pairs which can be selected from &,, £2, €3, i-€., 


1» £23 be Ea 3 £1 o3- 


Therefore, subtracting all terms involving double correlations from 
(1.44), we define the triple correlation K[é£,, €2,&3) by the formula 


K(é, by, 3] = <b b2&5> ea <é> K{é,, 3] a <&> Kfé,, &3] 1 


(1.46) 
— (E> KlEs, &2] — (Er) (£2) <Ea> - 
Using (1.41), we can write (1.46) in the form 
K{é,, ba, &3] = <é1826s> 7 <é> <bg3> Eas <b> <b183> 
(1.47) 


— (a> (E182) + 2¢E> (a> <Ea> - 
In the case where all three random variables coincide, the triple 
correlations (1.47) reduces to the third cumulant 
K{é, &, £1 = ky. (1.48) 


Moreover, by analogy with (1.43), we can introduce the triple 
correlation coefficient 


Ry = K(Ep £2, Ga] (KfEi, f15 £1] K[ 2) far £2] K[Ea, fa» Ss)}- 1. (1.49) 


To define the fourth-order correlation for four random variables, 
we have to subtract all possible double and triple correlations from 
the difference 


Exbabsta> — (61> fa) (Ea) (bad - (1.50) 
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The number of terms to be subtracted is determined by the number 
of possible combinations of the elements £,, £5, £5, €, arranged in 
groups of one, two or three elements. The formulas for higher- 
ordér correlations become more complicated as the order is 
increased. 

The most concise expression for mean values of products of 
several random variables can be written in terms of the charac- 
teristic function 


‘ Ol ty ones Me) = Cexp Hayky + -- + rEr)> (1-51) 


In fact, we have 
4 


: Giga ee) 


TF Guy... Ou, 62) 


Uy. MeO 


To obtain the corresponding formula for the multiple correlation 
of ofder r, we need only replace the characteristic function by its 
logatithm in (1.52): 

1 


Flay fe] = Fe PM te) . (183) 


Uy. ete 


3. Random Functions 


Next, we consider a random function ¢(t) of a single real argument 
t (she time) which varies over the interval [0, 7], say. The random 
function &(t) will be regarded as given if we know stable experi- 
mental conditions which allow us to obtain realizations of £(#) in the 
interval [0, 7]. For example, such a realization might be a phofo- 
graph (or oscillogram) of the random function, whose form depends 
on time. 

If we take r fixed values ¢,, ... t, from the interval [0, 7], where 7, 
t,..,¢, are arbitrary, then the values £(¢,), ..., &(t,) constitute a 
family of random variables. If the random function is completely 
specified, these random variables are also completely specified, 
for arbitrary 7, t,, ..., t,. Therefore, we can regard the corresponding 
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probability density w[é(t,), ..., &(t,)] as specified. This probability 
density is a function of 7, t, ..., £,, and can be written as 


Bf, — E(ty)] -- Se, ~ ECL = wy( yy ony Neh bpp voy be) (1.54) 


Each probability density (1.54), regarded as a function of #4, -.., Xp, 
satisfies the usual requirements imposed on probability densities, 
i.e., nonnegativity and the normalization condition. At the same 
time, as functions of ¢,,...,t,, the probability densities (1.54) 
satisfy certain additional conditions, which are a consequence of its 
definition. These are the symmetry condition 


Wy eney Kreg coey ty coed coy Lgey coy Ley vee) = Wyleeey Mey cong Mygy ceed coy bey snes Lay conde 
and the compatibility condition 


W(X yy very Ky} by woes be) = f eran weey Sp ges by coer Eng) DX pg one Urry + 


(1.56) 


Wecan consider a random function &(t) to be specified if the prob- 
ability densities (1.54) are specified for all 7, no matter how large. 
By choosing the points of division ¢,, t,,...,¢, sufficiently close 
together, we can replace ¢(t) by a sequence of random variables 
&(t,), €(t2), »--, €(t,), With an accuracy which is sufficient for all 
practical purposes. Thus, a random function £(¢) can be charac- 
terized by an infinite sequence of probability densities 


w,(x3t), Wal Hy, ei ty te), oes (1.57) 


which satisfy all the necessary requirements. 

If we know the first r probability densities, we can discard the 
probability densities w,,...,@,y, since they contain no extra 
information not already contained in the probability density 
W(X, «+4, Xp} ty, .-) t,). In this regard, it should be noted that the 
ideal state of affairs would be to describe a random process by a 
single probability density of the largest possible order, if such 
existed, but since the argument ¢ is continuous, there is no 
probability density of maximum finite order. However, it is 
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dothetimes possible to introduce a probability functional W[£(t)], 
which defines a “continuous probability density,” i.e., defines a 
probability density in an appropriate function space. 

instead of the probability densities (1.57), we can give the 
foliowing sequence of characteristic functions: 


O(uys 1) = Cent, 


(1.58) 
Og(uty, ys ty, te) = Ceteatltn+inetttady , 


Specifying (1.58) is equivalent to specifying the sequence of 
distributions (1.57). Here, as before, we need only retain the 
characteristic function of highest order (if there is one), since all 
characteristic functions of lower orders can be obtained from the 
one of highest order by setting certain arguments equal to zero: 


‘ (Ug, coer Uys bpp osey by) = Opa n(Myy vues Moy Oy oop OF bas ones Lege) (1.59) 


By increasing the order 7, thereby approaching the ‘‘most complete 

4 y app’ 8 P 
‘:Haracteristic function,’ containing as much information as 
pessible about £(¢), we obtain in the limit the characteristic 
sunctional 


ae @{u(ty) = (exp [ef ue) (rat) ) (1.60) 


which completely describes the random process é(t). In this 
Yormula, the integral in the exponential extends over the whole 
‘domain of definition of the random function. The characteristic 
functions (1.58) can be obtained from (1.60) by choosing argument 
‘fulnctions of the form 

an 


. u(t) = Y) ua8(t — ta) - (1.61) 

{In its general form, a random process‘ cannot be described by a 
fidite number of functions of a finite number of variables, and 
hence a random process is either characterized by an infinite 


‘The term random process (or simply process) is used as a synonym for a 
raadom function of time. 
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sequence of functions or by a functional. Instead of the sequences 

(1.57) and (1.58), it is preferable to consider a sequence of functions 

such that the functions of higher order do not repeat the informa- 

tion contained in the preceding functions, but instead carry only 

new information. Therefore, we now introduce another method of 

describing random processes, which has this and other advantages. 
The mean values 


CECE) os Ete) = Meta» worst)» (1.62) 


regarded as functions of t,, .,., t,, are called moment functions, and 
the infinite sequence of moment functions 


m(t)), my{ty ty) , malty, tay tg), o (1.63) 


gives an exhaustive description of the random process £(t). It is 
even better to characterize the random process £(t) by the sequence 
of correlation functions 


y(t) , Rot, t) ’ hts, to, ts) Nan se 59) (al 64) 


which are defined as the multiple correlations 


1 


Re(tyy ous te) = K{ECty) 4 E(t)] (1,65) 


regarded as functions of the times ¢,,...,¢,. The correlation 
functions, just like the moment functions, are symmetric functions 
of their arguments. As the order increases, the physical significance 
of multiple correlations decreases, and hence the first few functions 
in the sequence (1.64) are the most important. This is the advantage 
of describing a random process by its correlation functions, The 
vast majority of random processes encountered in radio physics 
have the property that when the times ¢,, ..., f, are moved apart, 
the correlation between the corresponding values of the process 
falls off, i,e., the correlation functions (1.65) go to zero (for 7 > 1), 
which is very convenient. 

If we know the moment functions or the correlation functions, 
we can find the other characteristics of a random process, in 
particular, its characteristic functions or probability densities. In 
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sy] . . a . 

fact, using (1.52) we can write the characteristic function as a 
rpultidimensional Taylor’s series: 

id & 
eB yoy hes tay voy be) = + 2 3, mite sees ty) Ma ss Hy» (1.66) 
{ 

Jn just the same way, using (1.53), we can express the charac- 
teristic function in terms of the correlation functions: 


\ 2 go oF 
P(t, voor ys bys vey Ep) = EXP fy + > Regltay soe by) Ma oes Uyt » (1.67) 
s=1 Oye Wend 


It is important to note that (1.67) does not change its form if we 
choose different times ¢), ..., ¢, or change the number r. Using this 
fact, we can write the characteristic functional (1.60) as 


t 
O{u(t)] = exp > 5 fiom f Aeltas oy ta) ty) a(t) dts ee dtsl. (1.68) 


in Chap. 3, we shall show how to go from characteristic functions 
ho probability densities. 

Because of the special role played by moment functions and 
‘correlation functions, we give the following formulas relating 
‘them: 


: m(t,) = A(t), 

; malty, te) = Reltyy te) + Filth) Alte), 

: tag ty tay ty) = R(t as toy ty) + 3{Rr(E) Kolbas ta) + Aa(ta) Aa(te) alta) » 

# ma tay tay tay ta) = Belt ay ty tay ta) + 3{halt ar fy) hata, tbe (1.69) 
i + Af Re(t1) Realtor tar ta)}s + 6€Fi(t1) Rate) Balto» t4)}s 

‘ t Ry(ty) Ri(te) Rats) Ai(ea) » 


Gann tetenenenee 
i Here, the symbol {...}, denotes the operation of symmetrizing the 
expression in brackets with respect to all its arguments. If the 
‘function to be symmetrized depends on arguments and has no 


symmetry at all, this means that we have to carry out all 2! possible 
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permutations of the arguments and then take the arithmetic mean 
of the resulting 7! terms. However, if the function has symmetry 
with respect to some of its arguments, then among these n! terms, 
there will be some that are identical. Therefore, we need not 
calculate the arithmetic mean of all the permuted terms, but only 
of those which are different. It is interesting to note that the 
coefficients appearing before each symmetrized expression in the 
formulas (1.69) are equal to the total number of permuted terms 
which are not identically equal. To form the arithmetic mean of the 
terms which are not identically equal, we have to divide the sum 
of these terms by a number which just equals the factor standing in 
front of the sum. Thus, all that remains in (1.69) are sums over 
those permutations of the arguments which lead to terms that do 
not coincide. 

The system (1.69) can be solved step by step for the correlation 
functions in terms of the moment functions. It should be noted 
that the correlation functions are not the same as the central 
moments. The difference begins with ky. Thus, if we set k, = 0 
in (1.69), we find that m,, the central moment of order four, differs 
from k, by the term 


mig(tay toy tay La) — Rally» toy tay ta) = Rol tay te) Rolta ta) (1.70) 


+ Reofty, t5) Ro(tey ty) 4+ Roltay ta) Falter ta) - 


CHAPTER 2 


Stationary Random Processes 
and Spectral Densities 


1. Basic Concepts 


' A random process is said to be stationary (in the strict sense) if its 
statistical characteristics are invariant under time shifts, i.e. if 
they remain the same when ¢ is replaced by ¢ + a, where a is 
arbitrary, Then, the probability densities w,(&, «5 Eni tay. tn), 
together with the moment and correlation functions m,(t,, ...) tn) 
and Ra(ty, ..., t,) do not depend on the absolute position of the 
points ¢,,...,¢, on the time axis, but only on their relative con- 
figuration, i.e., only on the time differences t, — ty, .... ta — ty. 
For example, consider the correlation function &, (¢,, ..., t,). By the 
definition of stationarity, we have 


Reg(tys oy tn) = Ry(ty + @, vr te $a). (2.1) 
Choosing a = —t, and writing 
Reu(Oy te — bys ooey ty — t1) = Ri(ty — tyy oy te — ty) 
we find that 
Realty cory bn) = Ralts — ty oe ty — (2.2) 


‘Thus, the correlation functions of a stationary process depend 
oaly on the time differences, as asserted. In particular, it follows 
that the mean value k,(t,) of a stationary process [equal to k,(0)} 
must be a constant, which we denote by m: 


k(t) = mit) =m. (2.3) 
21 
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Similarly, the correlation function ,(¢,, t2) is a function of t, — ¢,, 
for which we introduce the special notation 


Re(t, te) = (tp —t1), (2.4) 
where &(r) = k(—7). We can also write A(r) in the form 
R(r) = oF R(z), (2.5) 


where o = ofé(t)] = VA(O) is the standard deviation of £(#), and 
R(t) = R(r)/o? is the dimensionless, normalized correlation 
coefficient (1.43), with the property that R(O) = 1. We shall not 
introduce any special notation for the higher-order correlation 
functions, since they are not as important. It should be noted that 
when all the arguments of the third-order and fourth-order 
correlation functions coincide, we obtain the cumulants &, and ky, 
appearing in the definition (1.29) of the coefficients of asymmetry 
and excess: 


ya = °R:(0,0), ve = o *h,(0, 0, 0). (2.6) 


In what follows, we shall often use the concept of the correlation 
time, defined by the relation 


ee fi R(t) | dr =o [lee Lar. (2.1) 


The correlation time tcor gives some idea of the size of the time 
interval over which correlation extends between values of the 
process &(t). In fact, we can neglect correlation between values of 
the process whose time separation is considerably greater than 
Ter. We also define the quantity 


K= J R(t) dr, (2.8) 
which we call the intensity coefficient of the random process £(t). 


This quantity plays an important role in the theory of Markov 
Processes. 
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« Next, we introduce the important concept of the (power) spectral 
density S[é; w] of a stationary random process £(t): 


: S[és a] = 2 f 4 eardee,) de = 4 f * 00s wrét, dr. (2.9) 


Here, and subsequently, the subscript 7 on a function like é,, in 
the presence of a similar function like ¢ without a subscript, denotes 
h shift of the argument by an amount 1, i.e, £, = &(¢ + 7), while 
€ = &(t). For a process with zero mean value, S[é;w] is the 
Fourier transform of the correlation function: 

S[g a] =2f” Se Me) dr. (2.10) 


In the general case, where the mean value (> = m is not zero, a 
similar formula holds after subtracting m from £: 


SE — m; wo] = 2x(w) = ay eé** k(n) dr. (2.11) 


Then, the spectral density S[£; ] contains a delta function term at 
zero frequency. In fact, substituting (££,) = k(r) + m? into (2.9), 
we obtain 

, SlEso] = S[E — mj w] + 4rmB(w) . (2.12) 


It follows from (2.8) and (2.11) that the relation between the 
intensity coefficient and the spectral density is 


y K =4S[E—m;0]. (2.13) 
_ If we take the inverse transform of (2.11), we find that 
y= Ef x evios S{E — m; cw] deo 


Ps (2.14) 
=+f cos wr S[E — m;w] dw. 
0 


Th particular, this gives the formula 
t 


i o%(€) = AO) = = f y S[E — mw] deo (2.15) 
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for the variance of £. It can be shown that the spectral density is 

nonnegative for all w, and hence can be interpreted physically as the 

distribution over frequency of the power of the process £(¢). 
Inverting (2.9), we obtain 


<)> = + cos wr S[E; w] dw. (2.16) 

Sometimes it is convenient when calculating spectral densities to 
use techniques of operational calculus and the formula 

S[& wo] = 4ReL{( ££); iw] , (2.17) 


which follows from (2.9); here, L[f; p] denotes the Laplace trans- 
form 


Lifel= J erslnae. 


Example 1. Let &t) be an exponentially correlated random 
process, with correlation function 


R(r) = o%e-Attt, (2.18) 


Then, according to formulas (2.11) and (2.7), &(t) has the spectral 
density 
40°B 


SIE — Oso] = Tae Gib) 


and the correlation time 


(2.20) 


Example 2. If the correlation function has the form 
R(r) = ote! cos wer , (2.21) 


thé spectral density equals 


Ste — 50] = Ao fete 


pa (222) 


If the inequality 
B< w (2.23) 


holds, the spectral density is concentrated in a relatively narrow 
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frequency band w — w,~f, and we have the approximations! 
ot + af 2Bo 
StE =<} w] 480? (a? — wy? + 4B wu? a (e - wo)" + Be 

(2.24) 


2 





2.1 
Teor B 
The following table exhibits these and other commonly encoun- 
tered correlation functions, together with the corresponding 
spectral densities: 




















Taste I 

"No. R() S[é— mj wo] = 2x(w) 

1 K8(r) 2K 
, = 
og otePirl eae 

3 oe-1 apr? Va 2 en 1B? 
+—— 
5 2 2 2 
: 4 ateFlt! Cos wot 4Bo ae woth 


Gin wt — By at 





ws + # 


. é. 
a) ec Alt (cos wor + Gain | wor i) 4p (ata of — ARF apa? 





w 


&. 
—1alt gp ine Sens oe A A 
196 € {eos ” a . ) 48 (at — w? — f)? + 4fta® 

















; rf Be-t!*) — oe-B ltt 4aB(a + B) 0? 
p-a (w? + a7) (w? + 6%) 
¥ | 
F ooh Teife <piOife > p 


gS 4 
‘The symbol ~ means ‘‘is approximately equal to,’? while ~ means ‘‘is 


of the order of.” 
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2. Calculation of the Spectral Density 
by Using the Laplace Transform 


Consider the Laplace transform of a stationary random process 


(0): 
Lg Al = [er eeyat. (2.25) 


Multiplying this expression by its complex conjugate and averaging, 
we obtain 


CLEP] > = ff ere me cele) ate)» ded’, 


where the asterisk denotes the complex conjugate. Making the 
change of variables 


we find that 


CUB ALPS = J oe oP | (b+ 





74 rl cee ds. 


Integrating with respect to s, we obtain 


CIEE AID = io oe | sen ltl_2 poe rhCee bdr. 


- $ * I 
Multiplying by p -+ p* and comparing the result with (2.9), we 
arrive at the formula 


SUE; wo] = 2 lim (p + p*) <1 L{E: A]? » (2.26) 


which is sometimes useful for calculating spectral densities. 

Since the stationary process ¢(¢) does not grow at infinity, the 
Laplace transform (2.25) is defined for Re p > 0, and so is the 
quantity < | L[é; p] | >. If we set 
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then the limit p —-~ —zw corresponds to the limit « > 0, Therefore, 
instead of (2.26), we can write 


Atel tied |te3 





a . (2.27) 


3. The Random Spectrum and its Relation 
to the Spectral Density 


The spectrum y(w) of the process £(t) is defined by the formula? 
at ptt 
yw) = Tz f ot e(t) dt, (2.28) 


and is obviously a random function of the frequency w. 

x Strictly speaking, when £(t) is a stationary random process, the 
integral (2.28) diverges, so that (2.28) has only a formal meaning. 
In fact, (2.28) must then be interpreted as meaning that an ordinary 
equality is obtained if we apply a linear integral operator J,, to 
both sides of (2.28), and then interchange the order of integration 
in the right-hand side: 


! a y—2 | 
Joyo) = Fe J Uae a(t at. (2.29) 
Actually, this is the same way in which the familiar formal relation 
Rr) = x ig eit dy (2.30) 


is interpreted. The expressions (2.28) and (2.30) are so-called 
generalized functions. Such a function is regarded as specified if we 
know the integral of the product of the function with an arbitrary 
continuous function. Moreover, it is not necessary that the value of 
a generalized function be specified for every value of its argument. 
As is well known, one can deal with generalized functions like (2.28) 
and (2.30) in the same way as with ordinary functions. x 


2 The terms spectrum and spectral density are sometimes used as (rough) 
synonyms in the literature, but here their meanings are entirely different. 
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Multiplying (2.28) by its complex conjugate and averaging the 
result, we find that 


Ka) yw = = f J ertaetiore Caley ety dt dt’. (2.31) 
Making the change of variables 
1 , 
rat—t, t=5(t+t), 


or equivalently 
t=4-5, t=+5, 


and then integrating with respect to ty, we obtain 


(Yo) ¥*(w')) = Ko — w') i. exp jot, 





E62) dr, 


where we have used (2.30). According to (2,9), the remaining 
integral is just one half the spectral density S[é; w], and hence 


Ho) ¥*(w'y = 4 8(w — 0") SIE 2] (2.32) 


‘The inverse of the transformation (2.28) is 
ei) = af ett y(n) de (2.33) 
(t) = Vie ba Yu) do . z 


Since the process &(t) is real, taking the complex conjugate of 
(2.28), we easily obtain 


I*(w) = y(—e) - (2.34) 


We can then write (2.33) in the form 


kt) = af Re [ . et y(w) deo, (2.35) 


which is the customary way of writing oscillations in radio 
engineering (the method of complex amplitudes). Formula (2.35) 
represents the process £(é) as a superposition of harmonic oscilla- 
tions, where, in terms of the complex spectrum y(w), the oscillation 
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of frequency w has amplitude proportional to | y(w) | and initial 
phase equal to arg y(w). 

x The random spectrum y(w) appears only in the integrand of 
(2.35). Thus, to use this formula, relations of the type (2.29) are 
sufficient, and the strict equality (2.28) is not necessary. By the 
same token, to make our treatment more rigorous, we ought not to 
require that (2.31), (2.32) and (2.34) hold, but only that the corres- 
ponding formulas obtained by applying the integral operator J, 
hold. The use of integral operators is also justified physically, since 
experimentally we cannot obtain an exact harmonic, but can only 
isolate a sum of harmonics lying in a finite (albeit narrow) fre- 
quency band 4w. 

Choosing J,, to be the operation of integrating with respect to w 
from wy to wy + dw, with weight (1/4w)e'*, and writing 


a ! rs iwty 
Ye to) = ga J" em yu) do 
we obtain 
| — e-ett-ty) 


1 
Yow 6) = a J dot) 


enteull-te) €(t) dt, (2.36) 
according to (2.29). Carrying out similar integrations with respect 
tow and w’ in both sides of (2.32), we obtain 


yt 


1 dw 
<I Yo ty) > = Toy J SE w] dw . 


oe 
{f 4w is so small that the spectral density does not vary much in the 
interval from w» to w, + Mw, this formula reduces to 


I ¥(wo, 4) PY = sae SIE wd (2.37) 


The formula (2.37) shows the connection between the spectral 
density and the function Y(w, t)), which to a certain approximation 
veplaces the true spectrum y(w,)e*”*, If the bandwidth 4w is 
small, the factor 

1 — e~f4ere—t) 


iMuXt — to) 
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in (2.36) is slowly varying. This factor has the effect of restricting 
the region of integration in the divergent integral (2.28), and as a 
result, the influence of values of ¢ for which | t — t)|> !/dw 
becomes negligibly small. Of course, there are many other ways of 
restricting the region of integration in (2.28), corresponding to 
other forms of the integral operator J,, in (2.29). * 

Under actual conditions, one does not deal with the exact 
spectrum y(w). Thus, consider the function 


¥(a, 4) = « Glty — t) &(t) dt, (2.38) 


which is a good approximation to the spectrum y(w) if the function 
G(t) — t) is “narrow-band.” We shall regard the function G(t) — t) 
as being narrow-band if in the spectral expansion 


Cll) = He fee? gw)dw (few) do =1), 2.39) 





the function g(w) is appreciably different from zero only in a small 
interval 


| w — wy | ~ dw (dw < wo) (2.40) 
centered at w». This means that G(t, — #) is just the harmonic 
eot'o~ multiplied by a slowly varying function, which changes in 


a time of order !/4w. Substituting (2.39) into (2.38), changing the 
order of integration and taking account of (2.28), we have 


¥ (ey ta) = f ef gw) yw) deo. 
Multiplying (2.32) by g(w)g*(w’)e?-°""' and integrating, we obtain 
<1 ¥ (wo, to) P> = Ff SE] | g(w) [? de 


Suppose that the function g(w) is appreciably different from zero 
only in an interval (2.40) which is so narrow that S[£; w] practically 
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does not change there. Then, we can bring the function S[£; ] in 
front of the integral sign, obtaining 


1 ¥ (eos t6) P> = ¥ SLE; 9] f 1 gle) P deo. (2.41) 


_ According to formula (2.41), to obtain the spectral density it is 
necessary (in radio engineering language) to pass the random 
function £(t) through a narrow band linear filter followed by a 
quadratic detector, and then average the result. However, in this 
regard, it should be pointed out that any actual radio engineering 
device has a real impulse response G(ty — t). This means that g(w) 
has the property 
f &(—#) = g*(w), 
go that g(w) must be different from zero not only in the interval 
(2.40) but also in the symmetric interval | w + w)|~ 4w, where, 
because of its evenness, the spectral density S[£; w] takes the same 
values as in (2.40). Therefore, as before, we can bring S[é; w] in 
front of the integral sign. Thus, in the case of a real narrow-band 
process Y(wo, to), formula (2.41) remains the same. 


4, Time Averages of Stationary Processes 


, The so-called ergodic processes are an important subclass of the 
cass of stationary processes. If, we know a single realization of 
an ergodic process over the whole time axis —° <t <~, then by 
thaking time shifts, we can obtain an infinite statistical ensemble of 
realizations. Therefore, from one realization of an ergodic process, 
we can ascertain all of its statistical characteristics. 

For example, the mean value of &(t) can be calculated by the 
formula 


ee el 
m =in> J, &(t) dt. (2.42) 
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Of course, other formulas can also be used, in particular, the 
formula 


. 12 
m = lim watt + kA), (2.43) 


which replaces the integral (2.42) by a sum, where 4 is any positive 
number. To prove (2.42), consider the random variable 


2 f : et) dt, (2:44) 


which has mean value <m,> = m. As we shall now show, the 
variance Dm, goes to zero as T —> ~, 

Squaring (2.44), averaging the result and subtracting m?, we 
obtain 


Dmr =p f Me —1t')dtdt’. 
Making the change of variables 
r=t-t, t=$(t+’), 
we see that for a fixed value of 7, the limits of integration with 


respect to t, go from 4|7| to T — $|7|. Thus, integrating first 
with respect to fy and then with respect to 7, we obtain 


1 Ki 
Dm r= Fa f oT ID AG) ar (2.45) 
or because of (2.5), 
Dar = Ef S)mne 


Since Dm, is nonnegative, it follows that 


Dir = 75 f" (1-4) Rear | < Ef" (1 =F) Ro Lar 
<7 fro dr < 7% "1 Rey lar. 
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The last integral is the correlation time (2.7), and hence 
1 
: Dm < 20% er. ; (2.46) 


From (2.46) it is clear that if the stationary process €(t) has a 
finite correlation time and a finite variance o®? = (£%(t)) — m?, then 
4 Dmr—>0 as Tom, 


t 


This means that as T increases, the random variable m, converges 
to a quantity which is not random, i.e., the mean value m. The 
formula (2.46) also leads to the following rough estimate of the 
speed of convergence: 


Ramco dt—m|<o(2 Fae (2.47) 


The time average (2.44) converges at the same rate as the arithmetic 
njean of a family of N identically distributed, independent random 
variables ¢(t;), where N = T/2r¢o,. In fact, in order to simplify 
practical calculations of the mean value, it is convenient to go from 
the integral (2.42) to the sum (2.43) by choosing 4 F& 2r,¢o7. 

" Just as in the case of the mean value, we can use time averaging to 
calculate other statistical characteristics of ergodic processes. For 
example, forming the new stationary random function 


n(t) = (0) &E+ 7) (2.48) 
(where 7 is fixed), and applying formula (2.42) to y(t), we find that 


Eb) = im + Lf &(t) &t +7) dt, (2.49) 


a result which can be used when 7(t) has a finite correlation time. 

If we want to calculate the spectral density, then, according to 
(2,41), we have to time-average the square of the function ¥(w, ta). 
Both the process Y(w., to) and its square are stationary processes, 
regarded as functions of the time tp. Since Y(wo, to) is more narrow- 
band than the original process £(t), it has a longer correlation time, 
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whose order of magnitude is inversely proportional to the band- 
width dw. It is natural to assume that | ¥(w9, to) |? has approxima- 
tely the same correlation time: 


(Tem )y? H/4w. 


Applying formula (2.47) to the process | Y(wo, to) |?, we have 


D/¥ 22 
| Mont) P= pf 1 Monto Pdt| < (FEE). 2.50) 
or taking account of (2.41), 
in 2 2 ol YP) 
SE : d 
| ste a] = Tle dw xf | ¥ (oy to) | ta| ~ Sle dw Tis 
It can be shown that the estimate 
ol ¥ PP) ¥ *) ~ Sle; 
Slgtdw 
holds. This implies that 
. ord. 
S{E; a4] = Teta a tf | ¥(wo, t) Pat, (2.51) 


with a relative error of order (T4w)-1/2, We obtain the exact value of 
the spectral density if we pass to the limits T4w —» ~ and dw +0 
in (2.51). 

In making experimental measurements of the spectral density, 
it is not desirable to use a very small bandwidth 4w, since to obtain 
satisfactory accuracy the averaging time 7’ has to be very much 
greater than the time !/4w required for the narrow-band process to 
change appreciably. Of course, the o Blan of time averaging 
need not take the explicit form (1/T) J. . dt. In fact, as a rule, the 
integrators which perform the time averaging in practice have other 
weight functions. The important thing is that the integration time 
T should be large enough to satisfy the inequalities 


1 
T > Tears T>z 
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5. Cross-Correlation Functions 


Suppose we are given several random functions £(2), y(t), ..., 
which for full generality, we assume to be complex. If the processes 
are stationary, then their correlation functions 

. 


KH), Et) Kft), 0*(42)] (2.52) 


depend only on the time difference t, — t,. In addition to the 
‘orrelation functions (2.52), we are interested in cross correlations 
Letveea different random functions, e.g., €(#) and »(t). Two 
Stationary processes £(t) and »(t) are said to be stationarily correlated 
if the function 

: K(E(t.), 9 *(t2)) = Felt, — 4) (2.53) 


also depends only on the time difference t, — t;. The function 
(2.53) is called the cross-correlation function, as opposed to the 
functions (2.52), which are sometimes called autocorrelation 
functions. Unlike an autocorrelation function, a cross-correlation 
function need not be an even function of its argument t, — t,. 

If the mean values <£ and (n> equal zero, then the Fourier 
transform of (2.53) gives the cross-spectral density 


S{é, 954] ~2 f im e°*Eq*) dr, (2.54) 

which is not necessarily real, but has the property 
S*{E,n; 0] = S{n, 5 o]. (2.55) 

Moreover, for real processes, we have the relation 
S*{E.95 o)] = Slé, 95 —o] - (2.56) 


If we set » = € in (2.54), we obtain the spectral density of a single 
random function. In the notation of (2.54), the spectral density 
S[£; w] defined by formula (2.9) can be written as S[E, £5 w]. The 
inverse of the transformation (2.54) is 


(ént> = go : S{E, 93 w] #0" dw. (2.57) 
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We now consider the random spectra 


yee) = Fem [ete eae, oe 
Ieee) = em [et alt) dt, 


in terms of which we can write the processes £(t) and 7(t) as 
superpositions of harmonic oscillations, by using expressions like 
(2.33), or like (2.35) in the case of real processes. By the same 
argument as used to derive formula (2.32), we can easily obtain the 
formula 


iw) ¥X(w') = $F 8 — w') S[E, 75] (2.59) 


from (2.58) and (2.54). Next, we introduce the narrow-band random 
ptocesses 


Veo, 4) = [ eto gel) yew) deo = f Gully — 1) &(t) dt, 
(2.60) 

V (er to) = f emt gq(eo) Yon) deo = f Gully — 1) a(t) dt, 
which, like (2.38), replace the exact spectra. Here, the functions 
ke» Gy Bq» G, have the same meaning as before, but in the general 


case, they may not be the same for the processes £ and 7. Using 
(2.60) and (2.59), we easily find that 


<¥e(wes b6) ¥en 4)> = 4 f SE 95 o] ge) 8X(w) dw. (2.61) 


The presence of the functions g,, g, in (2.60) converts. £, 7 into 
the narrow-band signals Y,, Y,. If g, and g, have the form 


8A) = 8 (%) = gw), (2.62) 


where g(w) is a real function which is appreciably different from 
zero only in a narrow frequency band w — w, ~~ 4w, then 


CY {wy to) ¥#(Wo, to) = $51, 93 w] f gw) dw. (2.63) 


SEC. 5] STATIONARY PROCESSES AND SPECTRAL DENSITIES 37 


Therefore, to obtain the (complex) value of the cross-spectral 
density, we have to take the average of the complex quantity 


Y,¥* =|¥, || ¥, ee. (2.64) 


Here, we have to take into account not only the product | Y, | | ¥,, | 
cf the amplitudes of the narrow-band signals, but also their phase 
Gifference 4p, which determines the argument of the complex num- 
ber (2.64). The processes €(¢), (t) dealt with in radio engineering 
are real, and so are the transformations converting them into real 
ratrow-band processes Y,, Y,. This imposes the restrictions 


&(—-#) = 8), 8,(—#) = 7) (2.65) 
cng, and g,. If we set 


{ &(w) = 8,(%) = 8 |), 


where g is a real function, formula (2.6!) gives 


KY wo, to) Va(o, to)> = Re SE, 93 wo] i Bw) dw. (2.66) 


We see that forming the product of two real, narrow-band 
processes cannot give complete information about the cross- 
spectral density as a complex function. To complete the specifica- 
tion of S[£, 4; wo], we need a second pair of narrow-band processes, 
but only one function in the first pair need be changed. Writing 


ge) = gle), E(w) = —# (lot) sgne, 
we find from (2.61) that 
Veer te) Pole t)> = 1m S16, 45 oe] fee) dw. 2.67) 
Thus, to experimentally determine the cross-spectral density, it is 
not enough to form just one product of two narrow-band processes. 
Another product of the two processes must be formed, and before 


doing so, we have to use a phase inverter to introduce an extra phase 
sh‘ft (preferably of 90°) between the two processes. 


CHAPTER 3 


Gaussian and Non-Gaussian 
Random Processes. 
Quasi-Moment Functions 


1. Gaussian Processes and Their Probability Densities 


Suppose that all the correlation functions k,(t,, 22), Rg(ty, tes tg), +++ 
»f a random process £(t) are zero, except the first-order correlation 
fuhction k,(t,). Then, &(¢) is not random at all, and is said to be 
dejerministic, which means that all the realizations of £(t) are the 
same. Next, suppose that all the higher-order correlation functions 
of &(t) are zero, except the first-order and second-order correlation 
fuactions k,(t,) and k,(t;, tg). Then, the process £(t) is said to be 
Géussian (or normal). In this case, according to formula (1.67), we 
have the following expression for the characteristic function of the 
ntandom variables £(t,), ..., &(tn): 


, 


Bras Ma aye ta) = exp |i DA) ta — FD) Haast) nats]. G1) 
oo a,pal 
Using (3.1), we can find the probability density 
Wal E(ty), -o» E(tn)] 


by calculating the inverse transformation 
wal Eps os En) (3.2) 
é : I Oo oO Fe n 

= any is tay i exp |=! 2 abe| Oy (thy, oy Uns yy very ty) tty. dy, , 


og 39 
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where &, = &?,); this is just the multidimensional generalization 
of the integral (1.19). The expression 2, , ko(t,, t) 4M, which can 
be written as 


(| (eta) — Atel wa") 


is a positive definite quadratic form. Therefore, the absolute value 
of the characteristic function appearing in (3.2) certainly does not 
increase as uy, ..., u, get larger. This implies that regardless of the 
presence of the infinite limits of integration, the integral (3.2) 
exists, at least in the same sense as the integral (2.30). Actually, we 
can carry out the integration in (3.2) explicitly, obtaining 


Wa Eps er bn) = (27)? [det |] Ra(tas ta) II]-?? 


: @3) 
x emp |— BD dura — hilt (60 — Ait 


where det || k,(t,, ¢,) || is the determinant of the correlation matrix 


Felts ty) Rota te) 0 Pa(tas fy) 





U Rates te) =f” ee 
Roltn, 4) Rtn, ts) ee Raltny ta) 
and || 4,¢ || is the inverse of the correlation matrix, i.e., 
I] ap Il = Il Rates ta) I, Dy aaphaltor ty) = Bay (3.5) 
pel 


(5,, = | fora = y,8,, = Oif a 4 y). A probability density of the 
form (3.3) ts said to be Gaussian (or normal), and the random variables 
&,, «..) &, ate said to have a Gaussian (or normal) distribution. 

If &(¢) is stationary, and if we set n = |, the matrix (3.4) and its 
inverse each reduce to a single element, and in fact 


! ! 
1 Rip) 
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[cf. (2.5)]. Therefore, the one-dimensional probability density 
equals 





i 
w{é(t)] = read exp | — 52 (ete) ~ mp. (3.6) 

Setting n = 2, we find that the inverse of the correlation matrix is 
vy RO) — Rr) 
ba * A(r) \" _|BO —Be) FO — Fo) @.7) 
Vt Rr) RO) =k) _—_®@) ‘ : 

i HO) — Hr) RO) — RG) 
so that the two-dimensional probability density is 

ies ! 1 & —2RG) G6 + 4 
we(E,, fo) = nevicEes | 7g T — RX) 2). (3.8) 


' In certain problems where it is difficult to calculate integrals 
inyolving the probability density (3.8), it is convenient to expand 
(3:8) in a series of powers of the correlation coefficient R(r). To 
find such an expansion, we set n = 2, ky = m = 0 in formulas 
(3.2) and (3.1), obtaining 


(3.9) 
— Fuh + 2R() ue + a cy 


ml &) = ra is Vs exp } — ime, — iu’, 


Using the expansion 
I 





; eno Rusu, — > & ud (o?Ruyu,)? , (3.10) 
i nd 


we find that 





wae &) = 3-1 [ge fw exp] ints — HY a] 
: = (3.11) 


3 x [ae wpe | inate — 8 au] 


42 GENERAL THEORY OF RANDOM PROCESSES [cu. 3 
Making the change of variables 
A= on, Ay = ot, 


and using the integral representation 
Fo+in(g) = or dP exp |—irs S + dd (3.12) 
for the derivatives 


FOr+l(z) = Y= & en 2/2 (3.13) 


of the probability integral 


F(z) = ioe us e722 dx 7 


we obtain the expansion? 
R? 
wy, &) = aoe (& ) Fon (= fe ) rr (3.14) 


Formula (3.14) can also be written in terms of Hermite polynomials. 

Similar expansions exist for the higher-order normal distribu- 
tions, i.e., for the normal distributions of order three, four, etc. 
For example, making series expansions like (3.10) of the exponen- 
tials 


ePRits, gO Ratt, gp O*Rigtitin, 
where 


Ry = Rh —b), Rey = Rtg — ts), Ras = R(t — 45); 


See H. Cramér, Methods of Mathematical Statistics, Princeton University 
Press, Princeton, N.J. (1946), p. 290, where the result in question is attri- 
buted to early papers by Stieltjes and Sheppard. 
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we can write the three-dimensional characteristic function in the 
form 
O3(u,, uy, Us) = Exp \-$ sui + a + ap Di Clr ar = (Rash)? 
3.15) 
x py (- a (Rast) 2 (1 rin Rusty)’ 
7 
{n calculating the Fourier transform of (3.15), each term with 


fixed values of p, g, 7 reduces to a product of three integrals of the 
type (3.12). Therefore, we find that 


wba, ba €2) (3.16) 
i a: - é, - by) ‘ by R’,RYR’. 
= Ft, Fotrsy (2) Fetaty { =a ) Fry { = ) aga 


24 

According to what has been said, the most important (and the 
myst time-consuming) step in finding a multidimensional probability 
density for a normal process consists in finding the matrix || agg || 
which is the inverse of the correlation matrix || ko(t,, tg) ||. The 
problem of calculating the inverse matrix is equivalent to the 
problem of solving the system of equations 


' Reo(ty, t1) 1 + Re(tys te) Yo + + Raltas te) In = Ms 
1s ey eae 6.17) 
Rol tay 11) Yi + Reltms 2) Yo + oe + Raltns te) In = Xn 


If we solve (3.17) with respect to 4, ..., Yn, We obtain expressions of 
the form 


; Ia = dows (a = I, wm). (3.18) 


Using (3.18), we can easily find the quadratic form 


Dull — hulta)] {fp — Aalte)] 


RTs 
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appearing in (3.3). In fact, we need only form the sum 


29{x] = De a DY ae%pckp (3.19) 
ank a,p=1 


and then set x, = & — Ay(t,). 

This method is applicable for any number of instants of time 
t,, .++) tay and remains meaningful even when the points 4), ..., t, lie 
arbitrarily close together. In this case, the finite probability density 
wa [E(ty), +») €(tn)] goes over into a probability functional. 


2. Conditional Correlation Functions of Gaussian Processes 


Suppose we know the values ¢(7)), ..., £(T,) of a random process 
&(t) at certain time instants Ty, ..., T,. This information changes the 
initial data concerning the random process, in particular, its 
correlation functions and probability densities. The new, modified 
correlation functions and probability densities are said to be 
conditional (or a posteriori), as opposed to the original quantities, 
which are said to be unconditional (or a priori). Qualitatively 
speaking, as a result of knowing certain values of a random process, 
we obtain a new “a posteriori random process,” which is character- 
ized by different statistical properties. It should be noted that if the 
a priori random process is normal, then so is the a posteriori random 
process. 

According to formula (1.39), the conditional probability density 
of the random variables &t,), ..., &(t,), given the values 
&(T)), ..., &(T,), is equal to 


wl E(t)» oy Ete) | (Ti) (TSI 
(3.20) 


= Fy treat) os Bb) Tas on (TN 


where 1/N is a normalization constant, which does not depend on 
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&(t1), »-) &(t,). The Fourier transform of (3.20) with respect to the 
variables £&(2,),..., (tp) gives the conditional characteristic function 


eomal tty ony ty) = + fo fox fi met) 


x Wy of E(t), sory Elte), &(T), oeey &(T,)) a&(t,) on d&(t,). 


Expressing the last integral in terms of the a priori characteristic 
function @,, ,(t1, .:, Up Oy,» 0,), Which is the (r + s)-dimensional 
Feurier transform of the joint probability density 


x Wes LEC) wy Ee) ET ys ons AT ON) » 


we find that 


! = 
Deoma(tis ++ u,) = wor! a J exp } ee o€(Ts){ 

(21) 
- X yg g(eeyy vey Myy Vz, ooey Ve) AV, ore Vy 


The factor | /N in (3.21) is determined by the condition 
Deona(0,-... 0) = 1 

: The calculation of the integral (3.21) is completely analogous to 
the calculation of the integral (3.2), The only difference is that there 
is now extra dependence on the numbers 4,...,4,, which can be 
regarded as parameters. According to (3.1), the characteristic 
function appearing in (3.2!) can be written in the form 


yh g(tdyy 000) Uys Vyy very Uy) = EXP |i Ault) ug + > k, (Tg) 
a=l a 


Sok Tr z z 
AD) alte te) watt + 22 DD halter Te) wats — ¥D) hl Tar Tp) vars}. 
a, pm awl B= apa 
(3.22) 


If we let || a,¢ || denote the matrix which is the inverse of the 
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correlation matrix || ko(7., T,) ||, then it follows from (3.21), by 
analogy with (3.3), that 





1 
Ocona(Uyy v0+y Uy) = Ww ep 


> ky(ta) uy — + Refteo ty) Ugus 
= BY a(Pe) ~ dT) — FS hdTe ty] 0.23) 
ape r 


x [&T—) — A(T) — ix hy(Tp to) us. 


The factor 
(Q2a)-r? [det || kof Ta, Tp) II? 


has been incorporated into the factor 1/N’, which is independent of 
Uy, cory My 

We now introduce the conditional correlation functions k,(t) and 
&,(t, t’), in terms of which the conditional characteristic function 
(3.23) can be written in the same form as (3.1), ie., 


ona (lay se» Me) = EXP ey Byte) ta — 4D) Falter ty) watt. (3.24) 
a a. a,pml 


Separately equating the coefficients of the linear and quadratic forms 
in (3.23) and (3.24), we obtain explicit expressions for k,(t) and 


Rt, t’): 


Bt) = b(t) +B halts Ta) al) — bl) 
- (3.28) 


y(t, t’) = k(t, ) — dks, Tx) daphe(T g, t’) » 
a, pal 


These expressions completely determine the a posteriori random 
process, just as in the case of any other Gaussian process.” 

2 Formula (3.25) can also be applied (with an obvious modification) to the 
case where we know a continuous realization of the random process on an 
entire interval, 
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'As an example, consider a stationary process with k,(t) = m, 
k(t, t') = oR(t — t’), and suppose we know the value of the 
‘process at some time t,. Then, according to (3.25), we have the 
following conditional mean value and conditional second-order 
certelation function: 


y(t) = m+ Ret — &) {&(4) — mm), 
E(t, ¢) = of RU — f) — R(t — ) Ry — 1) 


Setting ¢ = t’ in the last formula, we obtain the conditional 
‘veriance 


(3.26) 


i (Dé()leone = of! — RE — b)] . (3.27) 


Note that the a posteriori process is nonstationary. As t — t, > 0, 
the statistical scatter of the values of £(t) becomes progressively 
smaller, since R(t — t))-> 1 as t — tg->0. As |t — t,|—>~, the 
correlation coefficient R(t — t.)—>0, and any difference between 
the a priori and a posteriori processes disappears. 


‘3. Non-Gaussian Processes and Quasi-Moment Functions 


When the higher-order correlation functions g(t, ty, ts), 
‘Rg(ty» tay tg, t4),-.. are different from zero, we have a non-Gaussian 
or non-norinal) random process. In this case, the expression (3.1) is 
yno longer sufficient, and in (1.67) we must also take account of 
-ternis for which s > 2. Although the characteristic function of a 
;non-Gaussian process can immediately be written as an expansion 
: ‘of the form (1.67), the calculation of the corresponding probability 
‘densities i is difficult, since in general, the Fourier transform of 
.(1.67) cannot be calculated directly. 

‘In order to simplify the problem of finding the multidimensional 
distributions of a non-Gaussian process, we introduce the so-called 
“quasi-moment functions® 


: altar tay ts) > Baty» tes tay ta)y - (3.28) 


*P, I, Kuznetsov, R. L. Stratonovich and V. I. Tikhonov, Quasi-moment 
functions in the theory of random processes, Dokl. Akad. Nauk SSSR, 94, 615 
(1954); Theory of Prob. and Its Appl., English edition, 5, 80 (1960). 
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These functions occupy an intermediate position (so to speak) 
between the correlation functions and the moment functions, and, 
together with A,(t,) and ,(t;, te), they also completely characterize 
the process. By definition, the quasi-moment functions are con- 
nected with the correlation functions by the relation 


exp ag x fates ein big) aoe | 


=14 355 x, Bellas os by) Ba 


29am 


(3.29) 


which holds for any values of 1, t), ..., try 21) +) Zn Using (3.29) and 
(1.67), we can express the characteristic function in terms of 
quasi-moment functions: 


Bato ta to bn) = exp} it ta + DY altar) Ha 
(3.30) 


x fl +25 _ > pate wot) ta to} . 


yor eytoeml 

Formula (3.29) allows us to find explicit formulas relating the 
correlation functions and the quasi-moment functions. In fact, 
these formulas are the same as (1.69) if we set &,(t)=0 and 
k(t, t2) = 0. The lower-order quasi-moment functions bs, b,, 5, 
coincide with the corresponding correlation functions. The 
difference between the correlation functions and the quasi-moment 
functions begins with 5,: 


by = kg, =k, 6, =k, 


= he + 10{ksks}, 5 (3.31) 
= na + 35f{kgka}, - 


Here, the symbol {...}, denotes the symmetrizing operation, just as 


in (1.69). 
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si 
: To calculate the n-dimensional probability densities, we have to 
take the Fourier transform of the characteristic function (3.30): 

' 


tal wees Eni tay oer ta) 


=Qny* f- few | fetta {1 +35 > Bilt ta) Man He 
; (3.32) 
x exp [iD hte) ta + FS hate ty) wat} diy ed 
aal "a, fal 


Interchanging the operations of summation with respect to s and 
integration, we can write the last integral in the form 


"y 
WalSry oor Ens bay over bn) (3.33) 


: * {1 + 42, Balter on ta) (— x) mn(- x) a2 (Eys on Ga) » 


where w® is the probability density of a Gaussian process with the 
same functions &,(t), &,(t, t’) as the original non-Gaussian process. 
According to formula (3.3), we have 


tE fa) = a) et I Ast) NP exp |~ 4 & sere 
q (3.34) 
where 
E Xq = &,— Ay(ta)- 


We can also write the probability density (3.33) in terms of 
gentralized Hermite polynomials. Suppose we are given a quadratic 
form 


ax] = $ > apk ap (3.35) 
a, pal 


in several variables x,, x2, ..., X,. Then, with the coefficient matrix 
|| a, || we can associate the generalized Hermite polynomials 


Hilxl,  Hapl#l sr Ha olth (3.36) 
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defined by the formula 
Ha, ofa] = ert) (— ¥) Be x) e-ole), (3.37) 
If we write 
ye = yee) = EL 2 toot (3.38) 
a 1 
then, according to (3.37), we have 
H[*] = ya» 
Hapl*] = Yaa — ap» 
Hapyl*] = Ya¥ Dy — Paty — Fay p — Appa (3.39) 





A gpyal*] = VaI AV V8 — FapV V5 — Vay AV5 — Pad QV y — 2p))'a5 
— 2paVaVy — Fy eVaVp — Aaptys — Fay2p3 — Fas2 py » 


In terms of the generalized Hermite polynomials, we can write the 
probability density (3.33) in the form 


Wy (Say over Eni by voey tn) (3.40) 


AYES Polar ta) Hal) — AsO] wb on bd 


Thus, we see that from a knowledge of the quasi-moment func- 
tions, we can immediately write down the probability distributions 
of any order as an expansion involving generalized Hermite 
polynomials. The greatest difficulty encountered in doing this is 
finding the matrix || 4,,|| which is the inverse of the correlation 
matrix ||k,(t,, t,) ||. Only the first few terms of the expansion (3.40) 
play an important role, and if we restrict ourselves to a certain 
number of terms of low order, we can neglect the terms of higher 
order, 

In the special case where » = 1, we have a one-dimensional 
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a 
probability density. When the indices coincide, the polynomials 
(3.39) become ordinary Hermite polynomials, i.e., 
A, be) = af)?H,(aqt?y,) = ary", (at?) » (3.41) 
m times 


where 
d\m 
= ett —29/8 
H,,(2) = e*?* ( 3) ern, 


If we bear in mind that 


See bees sh 

au a “helt, t) - a 

inthe one-dimensional case, then, according to (3.40) and (3.41), 
we find that the probability density is 





wf6(0 = f+ pt, LOTAO] meron, 40) 
where 


“6, = 6,(t,..,t), w= TEC — exp |— sa (é- hort. 
ae: 

The expansion (3.42) is known as Edgeworth’s series. 

, Next, we consider the two-dimensional case (n = 2). Then, when 
ihe process is stationary, the matrix || a,, || is given by (3.7), and 
tha probability density w? has the form (3.8). In this case, the 
vartiables y, and y, appearing in (3.39) are 


1 x, — Rx, 1 —Rx, + 2 
A=GaoR kao: (M8) 





oa 


Ia the two-dimensional case, the indices in (3.39) can only take the 
values 1 and 2. If we write 


Ay. je. = Hum» (3.44) 
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then it follows from (3.39) that 

Ay = Heo =¥i- an, 

Ay, = Hay = Ya — M2» 

Hy = Hon = ¥2— 45 

Ay = Hoo = yj — 341» (3.45) 

, Ay, = Hey = ¥i¥2 — 2a — M92» 

Hyg = Hazy = 192 — 2422 — Aaa» 

Ayyg = Hog) = ¥2 — 342292 - 
We can write (3.45) as 
Hx) = ag (f-1), Hay = sya (i +R) Hoy = aux (a 1), 
A) = aa @-*) Hen = as (siz, + 2Ra,— 22), (3.46) 
Hay = ww (2:23 + 2Ra,— 2%), Hon = aa (% — 42), 


if we introduce the notation 


we =, 4 

Lane (3.47) 
— Rx, —Rx, + x, 

= omy, = 2, 4 = oy, = 


Combining terms in the expansion (3.40) which differ only by 
permutations of the indices | and 2, we can write the two-dimen- 
sional probability density as 


rods) = |L+ BD bam Ham (0) — AO] WHE 60» 
(3.48) 
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where 
tlimes  m limes 


Beimny = Orgm(tas eves bys bay eves te) 


:An expansion of this type is essentially an expansion in orthogonal 
‘polynomials. 

: As we have just seen, the quasi-moment functions are the 
coefficients in an expansion of the probability density in an infinite 
series of generalized Hermite polynomials. This is true for any 
jnumber of random variables ¢(¢,), ..., &(t,) corresponding to values 
of the original process. If the points 4), ...,t, are made arbitrarily 
‘close together, we obtain the corresponding expression for the 
tprobability functional. 


CHAPTER 4 


Markov Processes 
and Related Processes 


"1, Definition of a Markov Process. The Stochastic Equation 


Markov processes, or processes without aftereffect, are a 
convenient abstraction. Although the actual processes encountered 
in radio engineering are not exactly Markovian, it is sometimes a 
good approximation to regard them as being Markovian. This makes 
it possible to obtain many concrete results by using the effective 
mathematical methods of Markov process theory. We begin by con- 
sidering ideal point processes without aftereffect. Then, we look into 
the question of when, and in what sense, an actual process encoun- 
tered in radio engineering can be regarded as a Markov process. 
« Let x(t) be a random process, and let x(t,), ..., x(t,) be a set of 
its values at the consecutive time instants t, > t, >... > ty. 
Consider the conditional probability density of the value of x(t) at 
the most recent time ¢,: 

jy] = en) on Ct) 
eo[x(ty) | x(t2), --- a(t) = W,al(ly)y oor Ctn)] (4.1) 
The process x(t) is said to be a Markov process (or a process without 
aftereffect) if the conditional probability density (4.1) depends only 
on the last value x(t.) and not on the preceding values x(t), ..., x(t,) 
[ta > ty, +4) te > ty]. Of course, (4.1) can still depend on x(t,), 
t, t,, and hence for a Markov process, we can write 


(2, | Hey oer Hn) = Pesta Xe) (n> 2), (4.2) 
‘where = lh) say ty 


55 
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If we bear in mind that by definition, the conditional probabilities 
(4.1) corresponding to different values of are always connected by 
the compatibility relations 


J a ap en Ha) 00 as oor Bn) Bn = A004 | ay oor Mnaa) 100 oe Mncads 
(4.3) 


then it can be seen by substituting (4.2) into (4.3) that the same 
function p,,; (%, ¥2) appears in (4.2) for all 2. This function equals 
the conditional probability density 


Peyrg%a» 2) = (% | 2) (t, > te), (4.4) 


and is called the transition probability. The definition of conditional 
probability implies the formula 


WO(H yy ony Hy) = Wy | Hay oes Ay) W(%Qy | H gy -ney Hn) oo» WF na | Xn) WCFn)s 
(4.5) 


from which it follows by using (4.2) and (4.4) that in the case of 
Markov processes, the multidimensional probability densities factor 
into products of transition probabilities, i-e., 


UOC 20) Xa) = Pry a Xa) Ptgty( He» Xa) --+ Pty _yt(Hn—29 %) WH) -- (4.6) 


Thus, if we know the univariate probability distribution and 
transition probability of x(t), we can write any multivariate prob- 
ability distribution of #(#), i.e., the functions p,, (x, x") and w[x(t)] 
completely characterize a process without aftereffect. 

The transition probability has to satisfy certain conditions. In the 
first place, there is the normalization condition 


Jw) de = 1, (4.7) 


which follows from (4.4), say. Moreover, integrating the distri- 
bution (4.6) with respect to some intermediate value >(t,), 
1 <k <n, we obtain a probability density w,_, of lower order, 
which in turn satisfies an equation like (4.6). In fact, we have the 
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following integral equation, called the Smoluchowski equation (or 
thé Chapman-Kolmogorov equation): 
J Petre 0) Pela» %0) da = Pulte) (> >t) (48) 


To obtain (4.8), we need only consider the relation 


J eletts)s Uta), ats] dae) = welll), 2(¢0)] 49) 


and then write w, and wz in the form (4.6). 

Next, we study the time dependence of the one-dimensional 
probability density. Setting » = 2 in (4.6) and integrating with 
Tespect to x2, we obtain the equation 


wo(2s ty) = f Peon) 004 fa) de, 4.10) 


where 
w(x, t,) = wlx(t,)], ty > tg 


To convert (4.10) into a differential equation, we choose t, close to 
t,, Setting 


tp=t, t)=td4r, % =x, y= x, 
we can write (4.10) in the form 
402%) = f Per eee, #) (x) do. (4.11) 
. We now introduce the characteristic function 
O(us 2) = <elulerny = f emler=) pr, era) de, (4.12) 
of the random increment x, — x which occurs during the time 


interval [t,¢+ 7], given that x(t) = x. Substituting the inverse 
transform 


Peret(tn %) = +f e~iules—2) G(u; x) du (4.13) 
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into (4.11), we obtain 

ys eee J ferme (U5 x) du w(x) de (4.14 

(1) = 5 ; . 14) 


According to the usual formula (1.22), the characteristic function 
(4.12) equals 


@(u; x) = 1 +E mae) (4.15) 
in terms of the moments 
mx) = (x, — x) (4.16) 
of the increment x, — x. It follows that 
w(x.) = y + he ff enim (nye du m,(x) wa) de (4.17) 
However, since 


a J e-iulz,-z) (iu)* du= (- a) J e7fulz,—2) dy 


(4.18) 


we find that 
= 1 a): 
to) = wl) + 2 (— ge_) (mle) eo(a)]- (4.19) 
Dividing by 7 and passing to the limit r —> 0, we obtain 
: <1 Q\° 
6) = Yop (- a) [KO we, (4.20) 
where the overdot denotes differentiation with respect to ¢t, and 


K(x) = lim ™) , (4.21) 
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provided that the limits (4.21) exist.! Equation (4.20) is called the 
stochastic (or kinetic) equation. In the case where the sum of deriva- 
‘ives in the right-hand side of (4.20) has an infinite number of terms, 
the sum is equivalent to an integral operator, but otherwise (4.20) 
is a partial differential equation. 

According to (4.21), the moments m,(x) = <(x, — x)*) depend 
om + in the following way: 


! m,(x) = K,(x) 7 -+ O(7?), 
m,(x) = K,(x) + + O(7?), (4.22) 


Using the formulas (1.24), we can easily obtain completely analo- 
gous relations for the cumulants k,(x) of the random increment 
ty, x: 


kx) = Kx) +002) (6 = 1,2)... (4.23) 
If we define (t’; x) as the derivative 


assy = y= 20 sn (4.24) 





of the conditional random process x(t’) which takes the value 
xit) = x, then ; 
i +e 
tk = J &(t') dt’, (4.25) 
t 


4 
and the cumulants &,() can be expressed in terms of the correlation 
functions k,(t,, ..., t,)¢ of the process €(t): 


‘ 4 +t 
LRG = i o J, Re(tyy ones tee dt, dt, (s = 1,2,+.)- (4.26) 


gt should be noted that (4.19) and (4.20) have a simple operator inter- 
pretation. For example, (4.19) can be written in the form 


8 
wn, = e(iz.a)a, 


{ 
ii we agree that the second operator x in the function @ of the operators 
i(@/@x) and x always acts before the operator i(@/@x). 
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Comparing (4.26) and (4.23), we find that the correlation 
functions k,(t,, ..., t,), have delta-functions singularities of the form 


ety oon tale = K(x) 8(ty — ty) 8) —t) +A, (6 = 2,344), 
(4.27) 


where the A, denote other possible functions which are less 
singular than the first term (with spectra that fall off faster as the 
frequency is increased). If we substitute (4.27) into (4.26) and 
integrate, the functions A, give rise to terms of order 1?, 7%, ... 
The functions K,(x) are called the intensity coefficients of the 
derivative (4.24), which correspond to the given value a{t) = ~ at 
the given time t. These intensity coefficients appear in the sto- 
chastic equation (4.20), and in nontrivial cases, at least one of 
them is different from zero. As the above considerations show, 
Markov processes are intimately related to certain special random 
processes whose correlation functions contain delta functions. 
Such random processes will be said to be delta-correlated. 


2. Delta-Correlated Processes 


Suppose we have a delta-correlated random process £(¢), described 
by the correlation functions 


Re(tyy woes ty) = Kits) (ty — ty) + (tr — 8.) (8 = 2,3, 64), (4.28) 


where the K,(t) are the intensity coefficients, which in general 
depend on the time. In the special case where the process 
é(t) is stationary, the coefficients K, are constant. Setting 
t, = t, =... = t, in (4.28) we see at once that a delta-correlated 
process has infinite cumulants, This shows that delta-correlated 
processes are not encountered in radio engineering practice. 
Nevertheless, such processes are sometimes useful as approximate 
or auxiliary working concepts. 

A special role is played by stationary Gaussian delta-correlated 
processes. Let ¢(t) be such a process, with zero mean value. Then, 
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among the functions (4.28), only the second-order correlation 
function 


heft, — te) = KyB(ty — t2) (4.29) 


is nonzero, ie, ky = Kj; = Ky=...=0, and according to 
(4.29) and (2.13), &(t) has the constant spectral density 


‘ S{é;o] =2K, w) = K,=K. (4.30) 
‘Therefore, &(t) will sometimes be called Gaussian white noise, or 
simply white noise, by analogy with white light, which has a 
constant spectral density. 

The spectral density of any process encountered in practice is 
only constant over a certain range, and there is always some cutoff 
frequency w, which is an upper bound for the frequencies appearing 
jin the spectral density of the process. This means that the for- 
mula [cf. (2.15)] 


DE =f” x(a) de (4.31) 
0 


does not give an infinite value for the variance of the process, but 
rather a finite value, whose order of magnitude is Kw,/7 ~ K/r cor. 
Replacing the actual process by a delta-correlated process means 
that this frequency w, is not explicitly taken into account. This is 
permissible if w, is considerably larger than all other frequencies 
which are important for the given system or problem. On the 
other hand, the correlation time 7,,, = I/w, will then be small, 
sin fact, smaller than ali other relevant time constants of the system. 
Under these conditions, we can make the substitution 


hate = 4) > (44 — 4) [ . hor) dr , (4.32) 


where the coefficient of the delta function is chosen in such a way 
that both sides of (4.32) give the same result when integrated with 
respect tot, — ty. 
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3. The Fokker-Planck Equation and the Kolmogorov Equation 


A Markov process is said to be continuous if its higher-order 
intensity coefficients K3, K,, ... equal zero. In this case, equation 
(4.20) takes the form 


ae) = — 2 Kew + 5A Kee], — 433) 


and is called the Fokker-Planck equation (or the diffusion equation) ® 
Introducing the probability current 


1 @ 
G(e) = Ki) o(e) — 5 Gy (Ke) o@)1 (4.34) 
we can write the Fokker-Planck equation in the form 
. , OG 
w+s=0, (4.35) 


which can be interpreted as the equation of conservation of prob- 
ability. The probability current (4.34) describes the amount of 
probability crossing the abscissa x in the positive direction per 
unit time. Consider the interval x, <x <x,, with end points 
xy and x,. Then G(x,)r is the amount of probability entering this 
interval in time 7 across the abscissa x, and G(x) is the amount of 
probability leaving the interval in time 7 across the abscissa x2. 
lf probability never disappears inside the interval x, <x < %,, 
end if there are no sources of probability inside x, < « < x,, then 


2 Concerning the physical meaning of the Fokker-Planck equation, see e.g., 
L.. S. Pontryagin, A. A. Andronov and A. A. Vitt, On the statistical analysis 
of dynamical systems, Zh. Eksper. Teor. Fiz., 3, 165 (1933), reprinted in 
A. A. Andronov, Selected Works, Izd. Akad. Nauk SSSR, Moscow (1956), 
p- 142; H. A. Kramers, Brownian motion in a field of force and the diffusion 
niodel of chemical reactions, Physica, 7, 284 (1940); P. I. Kuznetsov, 
K. L. Stratonovich and V. I. Tikhonov, Correlation functions in the theory 
of Brownian motion. Generalization of the Fokker-Planck equation, Zh. 
Fksper. Teor. Fiz., 26, 189 (1954). 
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‘the difference G(x,)r — G(x,)r is the increment of the total 
‘probability Je w(x) dx in the interval x; < x < x, ie., 


\ ili w,(x) dx — i w(x) dx = G(x.) 7 — G(x) 7. 
‘ 1 1 
: Dividing this equation by 7 and x, — x,, and passing to the limits 
it + 0, x. — x,» 0, we obtain the equation of conservation (4.35). 
Thus, just as the equation of diffusion or heat conduction involves 
a flow of mass or heat, the equation (4.35) involves a flow of 
.probability. In fact, from a mathematical point of view, all these 
yphenomena are described by an equation like (4.35) or (4.33). 
In every particular realization of a Markov process, the trajectory 
‘x(t) has a very complicated form, and can be thought of as being 
‘swept out by a “representative point,” which moves along the 
x-axis just like a Brownian particle or a particle undergoing 
diffusion. If we take a large number of realizations of the random 
Process, we obtain a large number of “representative points,” which 
move about in a random and erratic fashion. These points form a 
kind of “gas” which undergoes diffusion, and whose density 
at any point is proportional to the probability density. Each 
separate point represents a “molecule’’ of the ‘“‘gas,’”” and as they 
move about, “molecules” cannot be created or destroyed. Thus, 
in this model, we can talk about the ‘number of moving points” 
instead of the “amount of probability,” and we can treat fe w(x) dx 
-as the number of points in the interval x, << * < x,, or as the 
;amount of time that a single moving point spends in this interval. 
"Of course, in this language, we have to talk about the relative 
number of points in an interval (rather than the total number), or 
the relative amount of time spent in an interval by a single point, 
‘since these are the quantities corresponding to probabilities. 

In order to obtain solutions of the Fokker-Pianck equation (4.33), 
we have to supplement it with initial conditions and boundary 
conditions. If we specify an arbitrary initial distribution 


: (2, to) = w(x) 


i 
at some initial time t), we can find the subsequent evolution of this 


64 GENERAL THEORY OF RANDOM PROCESSES [cu. 4 


distribution [i.e., the function w(x, t) for t > t,] by solving equatior. 
(4.33). If the initial distribution is a delta function, so that 
w(x) = 5(x — x), then the resulting probability density is just 
the transition probability p,,(%, %)). Therefore, the transition prob- 
ability can be found as the solution of the equation 


Ps) _ 21K) pass) +4 Z Ks) pal 20)» (438) 


with the initial condition 
; Pigts(% %o) = B(x — %) - (4.37) 


It can be shown? that regarded as a function of xg and tg, the 
transition probability py,,(, x») satisfies another differential 
equation 


OP 14(% %0) OP in(*, ¥0) 1 8p 44,(%, Xo) 


a, aa, Kilo) ~ 5 “aaa 





Ky(%)- (4.38) 


This equation, which is the adjoint of equation (4.36), is called the 
Kolmogorov equation. In the case of a stationary random process, 
K(x) and K,(x) do not depend on ¢, and the transition probability 
Pract = Pr depends only on 7 and not on t. Then 


and the Kolmogorov equation (4.38) gives a second expression for 
the derivative op,/@r, i.e., 


2e — nfs) Xo) 2px 9) K (4, +} & %) Bad: (4.39) 


in addition to the expression for 0p,/ar given ay (4.36). 


3See e.g. A. T. Bharucha-Reid, Elements of the Theory of Markov 
Processes and Their Applications, McGraw-Hill Book Co., Inc., New York 
(1960), Chap. 3. The Kolmogorov equation is often called the backward 
equation, and the Focker-Planck equation the forward equation, for an 
evident reason. 
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Next, we consider the boundary conditions which have to be 
taken into account when solving the Fokker-Planck equation 
(4.33) or (4.36). If the function x(t) can take all possible values 
f-om —o to o, the Fokker-Planck equation is valid on the whole 
infinite x-axis, and then the boundary conditions take the form 
of conditions at +. Integrating (4.35) with respect to x from 
--oo tO ow, and bearing in mind that the normalization condition 
J w(x) dx = | is satisfied identically for all t, we easily see that the 
condition 


G(—e, 1) = G(e, 1) (4.40) 
must hold. However, in addition to (4.40), the stronger conditions 
G(—e, t) = Gle,t) = 0 (4.41) 

and 
—o, t) = Wo, t) = 0 (4.42) 


are usually satisfied. According to (4.41), the ‘representative 
foints” of the process cannot appear at infinity or leave at infinity. 
In cases where the function x(t) can only take bounded values 
lying in some interval 

yt Iw, (4.43) 


we only consider the Fokker-Pianck equation in this region, and the 
boundary conditions take the form 


G(x, t) =0, G(x, t) = 0. (4.44) 


According to (4.44), there is no flow of “representative points” 
across the boundary, i.e., no random trajectory can enter the 
region (4.43) by crossing the boundary, and every random trajectory 
terminates when it arrives at the boundary. 

. , Of course, depending on the problem under consideration, 
we can have other boundary conditions as well. For example, 
in Vol. II, Chap. 9, we shall consider a case where the 
boundary condition takes the form of a periodicity condition 
w(x + 27, t) = w(x, t). The question of boundary conditions is 
also approached differently in first-passage time problems, which 
will be discussed below. 
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4. Solutions of the Fokker-Planck Equation 


The probability density w(x, t) satisfying the Fokker-Planck 
equation (4.33) is uniquely determined by the prescribed initial 
and boundary conditions. If the coefficients K,(x) and K,(x) do 
not depend on the time, then, as time increases, the distribution 
w(x, t) usually approaches a stationary distribution w,{x), which 
does not depend on the initial distribution w(x, t,) or on the time ¢: 

(x) = 0. 
Setting w,, = 0 in (4.35), we find that 
G(x) = Gy = const , (4.45) 


ie., the probability current is a constant, independent of both 
x and t, According to (4.34), for a fixed value of G,,, equation 
(4.45) is a linear differential equation in w,;: 


- [Kxlx) w,(%)] — 2K, (x) wa() = — 264 (4.46) 


The general solution of (4.46) can be obtained by ordinary 
methods. If we write K,(x) w,{x) = v(x), then (4.46) becomes 


K, 
Bie 


Multiplying (4.47) by 


v= —2Gy.- (4.47) 





vol -2 fi 


and integrating with respect to x, we find that (4.47) has the 
solution 


u(x) = —2Gy, “op j2 is Re dy| dx’ + Cexp Pi = rae 


from which it follows that 


wa(2) = eohea el es ze o- ae * exp {2 f° 2 dy de’. 
; (4.48) 
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Here, C is an arbitrary constant of integration, and the limit of 
integration x, is also arbitrary. However, apart from G,,, there is 
essentially only one arbitrary constant in the solution (4.48), since 
changing x, is equivalent to changing C. The choice of the constant 
€ (or equivalently, of x,) is determined by the normalization 
gondition, while the probability current G,, is found from the 
boundary conditions. If we choose (4.41) or (4.44) as the boundary 
éonditions, then the expression (4.48) simplifies to 


: c Kyy) 
f wut) = egy HP j2 f. or dy. (4.49) 


Thus, from a knowledge of the functions K,(x) and K,(x), we can 
immediately write down the stationary distribution in terms of a 
quadrature. This shows the effectiveness of stochastic methods 
based on the Fokker-Planck equation. Unfortunately, investigation 
of “transient processes,” which involves calculation of the transition 
probability by solving the nonstationary equation (4.33), is a 
much more difficult problem. However, in certain special cases, 
tthe problem can be simplified, e.g., when K,(x) is a constant and 
1K,(x) = ax + 6 is a linear function of x; then, the process is 
Gaussian and can be studied by using the theory of Gaussian 
processes. 

In some instances, the nonstationary Fokker-Planck equation can 
be solved by applying the method of separation of variables, i.e., we 
look for a solution of the form 


w(x, t) = X(x) T(t). 
Then, dividing both sides of equation (4.33) by w(x, t), we obtain 


7657 — Zk) X@)1 +52 1K,@) Xe] X+@) = a, (4.50) 


where A is a constant. For certain prescribed boundary conditions 
(usually zero boundary conditions), the resulting equation 


£& (Ka) X(0)] — Z1Ky@) XO) 42X@) =0 (451) 
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has solutions consisting of a sequence of eigenfunctions X,(x), 
X,(x), X,(x), ..., with corresponding eigenvalues Ay, 4, Az, «.- 
Integrating (4.51) with respect to x, we easily obtain 


Nw J X,(*)dx =0, (4.52) 


if the difference in the currents 


OX) = ~4 2 KX a + KX 
evaluated at the limits of integration (i.e., at the boundaries) 
vanishes. It follows from (4.52) that either A,, = 0 or [X,,(x) dx = 0. 
The value A = 0 is actually an eigenvalue, and in fact, for \ = 0, 
equation (4.51) reduces to the stationary equation. Thus, the 
eigenfunction X, corresponding to Ay = 0 is just the stationary 
distribution (4.48) or (4.49) found above, i.e., 


Xo(x) = wy (2) - (4.53) 


The other eigenfunctions X,, satisfy the condition 
f X,,(x) dx =0. (4.54) 


[n terms of the eigenfunctions, we can write the solution of equation 
(4.33) in the form 


w(x, t) = T,X,(x) + > T,,€7 2a!" X (x) (4.55) 


where the coefficients 7,, are calculated by using the initial 
conditions. To calculate T,,, it is convenient to use the orthogonality 
of the eigenfunctions (with weight |/w,,), as expressed by the 
formula 


J Xale) Xn) a = Ban + (4.56) 


Vere assumed that A, 4 A, if m Kn.) 
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% To establish the orthogonality relation (4.56), we write (4.51) 
for two different eigenfunctions: 


‘ a a 
; Fy Cl*nl = AnXns 3g Cl%nl = Xn 
where 

GX] = —55 2 1K,X) + KX 


We multiply the first of these equations by X,/w,,and the second by 
XbltOop and then integrate with respect to x. The result is 


' ad 
f Xi 2 aX, 4 sical i XiXn > (4.57) 

a dx dx 
af Xm ju GLX] ge = On J Xnka a: (4.58) 


Next, we prove that the left-hand sides of equations (4.57) and 
(4.58) are equal. Consider the integral 


7] 1 e 7] 
I= fg 5 Glouflds = —5 fe zylKuoufldx + fees (Kueusl, 


which can be written in the form 


ts) a 
Dah f eK de — fe 2 hyd Lae — Ff of Z Kao ae 
a a 
: a J Kru ax a Jes [Kyw,,] dx. 
j 
The third and fifth terms vanish because of the fact that the 
stationary distribution w,; satisfies the equation 
i 
: 2Gfes] 
: ex 
y 
Moreover, it is clear from (4.46) that under the condition 


Giusy] =0, (4.59) 
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the second term equals 


a 
—2 J eK wn x dx, 
and can be combined with the fourth term. Therefore, we have 
T=—- Lf pk you oh Oe dx — J eky0nZ ae. 
Integrating the last equation by parts, we obtain 
lr @ 8 é 
b= 5 f Za Kengl fds + [ 5 [Krogh fede = [5 Gloyel fax, 


provided that 


af oe, 
3x ax = 
or 
Glu f] — Glenglf =0 (4.60) 


on the boundary of the region under consideration. 
If we now set gy = X,/W,y f = Xp/Wgy we see that the relation 


JX FOI S = f XQ Z OS (4.61) 


holds, i.e., the left-hand sides of (4.57) and (4.58) are equal, as 
asserted. For this to be true, it is sufficient that the condition 
(4.59) and at least one of the boundary conditions 


G[X] =0 or = - Q onthe boundary 
be satisfied. It follows from (4.57), (4.58) and (4.61) that 
dx 
On — a) f XmXn ze = 0, 


which proves that the eigenfunctions corresponding to different 
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rd 

cigenvalues are orthogonal, with weight I/w,,. Here, we assume 
that each eigenfunction is normalized in such a way that 

i 


fuse=i (m = 0, 1,2, +»). 


Thus, we have proved the orthogonality relation (4.56) in the case 
where there exists a stationary distribution w,, satisfying (4.59), 
and no two eigenfunctions have the same eigenvalue. In most cases 
encountered in practice, the system of functions X,,(x) is 
complete.* 

The coefficients 7, appearing in the expansion (4.55) can now be 
expressed in terms of the initial distribution w(x, t)). Setting 
t = ty in (4.55), multiplying both sides of the resulting equation 
by X,,/w.: and integrating with respect to x, we obtain 


Tm = f 20%, t6) Xn(2) (4.62) 


w ron 
where we have used the orthogonality relation (4.56). If the initial 
distribution is a delta function, i.e., if 


W(x, to) = B(x — x), 


then the distribution w(x,t) is just the transition probability 
Du,(% %o), and in this case, according to (4.55) and (4.62), we have 


Putts x0) = Y Ane) Xd) e-agtety (4.63) 


Wt(%) 


sBy using the transition probability (4.63) and the initial distri- 
‘bution, we can write probability densities of any order. When the 
initial distribution is stationary, the expression for the two- 
dimensional probability density w,(x, %) = Py.4,(% %o)@sA%) is 
especially concise: 


tos &4) = YY Kal) Xt) e->et (4.64) 
m=0 


cnt 
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In particular, this implies that the correlation function and spectral 
density of the random process y(t) = F[x(t)] are given by 





Ay(z) = Sie intl (4.65) 
m=1 
and 
Sys] = Dy a ; (4.66) 
where 
= J F(x) Xp(x) dx. (4.67) 


Example 1. Consider the equation 


K ow 
= gO) +> 55 


In this case, (4.51) takes the form 


ot ote + Zex) 44 ax=0 (0? = #5). (4.68) 


If we impose zero boundary conditions at x = + o, then, according 
to formulas (7.355.3) and (7.351) of Ryshik and Gradstein’s hand- 
book, equation (4.68) has eigenvalues A,/B = n (n = 0, 1, 2, ...) 
and eigenfunctions proportional to 


Fins) @) ; 


d 
Vv TE dz” 


Thus, if we make the appropriate choice of normalization constants, 


where 


Fa+b (3) = el, 





Xl) = Taken (2), = mB (4.69) 


n 


and (4.64) is identical with the expansion (3.14), where R = el"!, 


“I. M. Ryshik and I. S. Gradstein, Tables of Series, Products, and Integrals, 
VEB Deutscher Verlag der Wissenschaften, Berlin (1957), pp. 405, 404, 246. 
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“Next, we consider equation (4.68) for x > 0 (instead of 
+o <x <), with the boundary condition GLX],_) = 0. In this 
case, we can only keep the eigenfunctions (4.69) which have even 
indices, i.e., 

2 1 


Xu) = Saar Fm €), Aw =2mB, (4.70) 


and the expansion (4.64) now becomes 


10,(x, 9) = Fann € Fem (je e-tmplel, (4.71) 


23a a 
Example 2. Consider the equation 


= §l—a)] +4 a 


Which describes a Rayleigh process (see Chap. 7, Sec. 3). The 
aes equation for the eigenfunctions is 
o 


as x + 2 Gex)- eZ (%) 44x =0 (# = 4), 


d 
3 
which becomes 
4, 


ax _ + [PEORA, qa|*=0, 


Set (4.72) 


after we make the change of variables z = x?/20?. If we impose zero 
boundary conditions at x = Q and x = ow, then, according to 
formulas (7.302) and (7.142.1) of the handbook cited above, 
equation (4.72) has eigenvalues A, = 2nB (n = 0, 1, 2,...) and 
eigenfunctions proportional to 


ae*L, (2), 
where 
d* 
L,(2) = eT (2°) 


5]. M. Ryshik and I. S. Gradstein, op. cit., pp. 396, 381, 247. 
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are the Laguerre polynomials. Therefore, we have 


X,(x) = aacm™ L, ( = 


alt (4.73) 


and the expansion (4.64) takes the form 





HXq og — 1 xt che Seer e 
w,(x, %) = re « ‘bre Gls (sr)L, ( i ) entnsltl , (4.74) 


Example 3. Consider the somewhat more general equation 


= Sle + ae 415) 


with zero boundary conditions at x = 0 and x = ~, The corre- 
sponding equation for X is 











ax a 2y 04 X 
at E OM -8 a(S J) +pX=0, 
which takes the form 
a btn - (2B), $- 
Be Oe E sH]u=0, 4.76) 


where 44 = (2y/K) — 1, after we make the change of variables 


z= el . ee eX, 

Again using formulas (7.302) and (7.142.1) of Ryshik and Grad- 
stein’s tables, we find that equation (4.76) has eigenvalues A, = 28 
and eigenfunctions u,(z) proportional to 


gett e*LQn)(z) . 


Returning to the old variables and normalizing the eigenfunctions, 
we obtain 
1 Aetl 


x 
———— ee en? 180) . 
Val Pn + 2p + 1) PQe + 1) 2Hot+? Z i (ar) 


X,(2) = 
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Then, according to (4.64), we have the expansion 





: 2 [ao \M cat 28) /2o% ; Ba 
wel 0) = 4 ($3) er La P(n + 2p + 1) PQpn + 1) 


(4.77) 





x Lis ( os 7) Ege (#4) : 


which is a generalization of the expansions (4.71) and (4.74). 


5. The Multidimensional Fokker-Planck Equation 


The considerations given in Sec. 1 of this chapter can be 
immediately generalized to the case of a multidimensional Markov 
process, which consists of several random functions x,(t), ..., *»(t). 
Such a process is described by a transition probability 


: Pula), os Hm(2)s H(t’), ¥m(EV 


i terms of which we can write the multidimensional probability 
densities. This transition’ probability is just the conditional 
probability density 


: Dilla on Beatie og xh) = ee aie) fgg 


W(,, oy X) 

of the random variables x,(t) = *), ..., x(t) = x, at a general time 
t; given the fixed values x,(t') = x, ..., xm(t’) = x, at a previous 
time t’ < t. In the case of a continuous Markov process, the one- 
dimensional probability density w[x,(t), .... *m(t)] satisfies the 
multidimensional Fokker-Planck equation 


U(X, v0) Xm) = — ‘ = [Kata +++) Xm) W(%1, ++) Xm)] 
(4.79) 


a a, [Kapl* 1 +++) %m) O81, 0 Xn) 





@,p=1 
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where the intensity coefficients K, and K,, are defined by the 
formulas 
a 
K,= ting ~ ax — 4), 
F (4.80) 
Kus = ling” <@ar — *,) (pr aa %g)> , 


by analogy with (4.21). 
In a special case, which might be called the case of ‘‘isotropic 
fluctuations,” the matrix || K,, || has the form 


Kag = K8ap, (481) 


and then the Fokker-Planck equation (4.79) can be written in the 
form 





7 w 8G, 
w= -2 Be, (482) 
where 
Gals) = Kats) 1s) — 5 3 [K(H) (6)] (483) 


are the components of a probability current vector G = (Gy, ..., Gn) 
in m-dimensional space (x denotes the set of coordinates x, ..., x»). 
As can be seen from (4.82), the stationary probability density w,, 
satisfies the equation 





pee (4.84) 
gal 


However, if m > 1, the probability current G does not have to 
vanish inside the region R under consideration, even if G satisfies 
zero boundary conditions 


G,(x)=0 («@=1,...,m) (485) 


on the boundary of R, since rotational probability flows can occur. 
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In. fact, the current G vanishes in the whole region R, i.e., 


K,(#) 42) — 42 [K(@) wylx)] = 0 (4.86) 


i a 
: 2 Ox, 


only in a special case which we call the potential case. 
If we substitute 


K(x) wax) = 0) (487) 
into (4,86), we find that 
au Ka 
ae 2 x’ (4.88) 


This shows that in the potential case, the quantities 


K, a(%) 
K(@) 





are the components of the gradient of a certain function, and hence 
satisfy the conditions 
a (Ky\_ @ | Ke 

tay (BE) = ey FE): (4.89) 
If the conditions (4.89) are met, the function U is given by the line 
integral 
em 2 
oe Kides + + Kmdtn) + C, (4.90) 


where aj, ..., 2, and the additive constant C are arbitrary. In terms 
of the function U, the stationary probability density is given by the 
formula 


Walt orn) = RGU, (ASN) 


where the additive constant in (4.90) is determined from the 
normalization condition 


if aes J mden wey Xyq) Oy». Xm = 1. 
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The solution (4.90) and (4.91) is an obvious generalization of 
formula (4.49). If the “potential conditions” (4.89) [i.e., the 
conditions corresponding to the potential case] are satisfied, we 
can always write down a stationary distribution (4.91) such that 
the Fokker-Planck equation is satisfied and the current (4.83) 
satisfies the zerc boundary conditions (4.85). If the boundary 
conditions are different, or if the potential conditions (4.89) are 
not met, the problem of finding the stationary distribution 
becomes much more complicated, 

If the coefficient K does not depend on xj, ..., x, the conditions 
(4.89) reduce to the conditions 


aK, 0K, 
axg = Ox,’ 








(4.92) 


which are to be expected if we regard the K, as components of a 
force which can be derived from a potential. 

* Next, we generalize the above considerations to the case where 
the matrix || K,,|| is arbitrary, rather than a multiple of the 
unit matrix, as in (4.81). The absence of probability current is now 
expressed by the conditions 


I a 
G, = Ky — 5) z Oxy [Kags =0. (4.93) 


Setting w,, = e~9 in (4.93), we find that 


m 


xx ae Es = —2K,. (4.94) 





Solving these equations for the derivatives 9U/x,, we obtain 


—* 








ae 
ye ae = 2> AyaKa ’ (4.95) 


where || A,, || is the inverse of the matrix || K,, Il, i-¢., 


D> AyKag = 8p 
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It fallows from (4.95) that the potential conditions now take the 
forra 


2a daly Bean) = Baal ean) 
(y,6 = 1... m). 


If these conditions are met, and if there is no flow. of probability 
through the boundary of the region R, then the assumption that the 
probability current vanishes everywhere in R is justified, and we can 
find the stationary probability density by calculating the potential 
function 'U from its gradient (4.95). To do so, it is necessary that 
the matrix || K,, || be nonsingular so that it has an inverse.« 
Regardless of whether the potential conditions are met, the 
solution of the Fokker-Planck equation can also be simplified in the 
case where K,,...,.K,, are linear functions of the arguments 
Xy ++) Xm, and the functions K,, are independent of 4, ..., %m- 





(4.96) 


6. First-Passage Time Problems 
1 


The apparatus of Markov process theory enables us to solve 
many problems involving processes which eventually terminate; 
this category includes first-passage time problems, which we shall 
cxamine only in the one-dimensional case, where the region under 
consideration is an interval x, << x <x, Let x(t) be a Markov 
process, and suppose we are given the initial distribution 


2 
a (4%) = x) (f le) dt = 1) (4.97) 
ya 
of ‘the values x(t). We are interested in the time ¢,,, which it takes 
the process x(t) to first reach the boundaries x = x, or x = %y, 
This so-called first passage takes place at times which vary from 


realization to realization, so that the first-passage time 


: Trp = by — by (4.98) 
‘is a random variable. 
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We now show how to calculate the mean first-passage time 
<typ>» confining ourselves to the case where the coefficients K, and 
K, in the Fokker-Planck equation (4.33) are independent of time. 
Excluding from consideration any realization of the random process 
x(t) as soon as it takes the boundary values x, or x, for the first time, 
we describe the remaining realizations by a probability density 
w(x, t) such that 

AP = w(x, t) Ax + O[(4x)'] (4.99) 


is the probability that at time t, the process x(t) takes a value in the 
interval [x, x + 4x] without ever having reached the boundary 
during the entire time interval [t,, t]. Then, the integral 


wey =f % w(x, t) dex (4.100) 


gives the probability that x(t) never reaches the boundary during the 
time interval [t,, ¢]. Initially, when no realization has yet managed 
to reach the boundary, the probability density w(x, t) in (4.99) is 
the same as the original density (4.97), so that 


WP) =1. (4.101) 


At subsequent times, the normalization condition is no longer 
valid, since more and more realizations are excluded from con- 
sideration as a result of having reached the boundary. Sooner or 
later, all the trajectories arrive at the boundary, and hence 


We) =0. (4.102) 


Inside the interval [-,, x,], the behavior of the probability density 
w(x, t) is described by the usual Fokker-Planck equation (4.33) 
[or by the conservation equation (4.35)], since trajectories cannot 
terminate inside [x,,x,]. In fact, trajectories are excluded from 
consideration only when the boundary is reached, and there is a 
nonzero probability current at the boundary, corresponding to a 
flow of “representative points” which are “absorbed” by the 
boundary. Therefore, the boundary conditions have to be altered 
in a basic way, as now described. We have already noted that 
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the derivative dx(t)/dt of a Markov process has infinite variance. 
This means that the instantaneous velocity of a “representative 
point” (i.e, a point describing a trajectory of the process) is 
infinite. However, only a finite displacement occurs in a finite time, 
which is explained by the fact the velocity of the representative 
point changes its sign with “infinite frequency,” while the point 
rhoves in both directions. Thus, if the random function takes the 
value x at time ¢, then in the very recent past it must have taken 
values both larger than x and smaller than x. Therefore, at time 
t> t,, there are practically no trajectories near the boundary 
which have not yet touched the boundary. But it is just these 
trajectories which are described by the probability density w(x, t), 
and hence w(x, t) vanishes on the boundary: 


w(x,,t) = wept) =0 (t>t). (4.103) 


Zero boundary conditions like these are typical of problems 
involving first-passage times. 

The initial and boundary conditions (4.97) and (4.103) uniquely 
determine w(x, t) as a solution of the Fokker-Planck equation (4.33). 
After calculating w(x, t), we can find the probability 


W(t.) — W(t) = 1 — Wt) (4.104) 


that the boundary is first reached during the time interval [to, ¢]. 
Nifferentiating (4.104), we obtain the probability density of the 
first-passage time (4.98): 

dw 
i (77) = — hh (to + Thy) + (4.105) 


Using (4.105), we find that the mean first-passage time is 

t 20 20 

ky =—f dW +7) =] Wt, (4.106) 
Typ A) 


where we have integrated by parts and used (4.102). If we choose a 
Sixed coordinate value x, as the initial condition, so that 
t 


w(x) = 5(x — x9), 
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then w(x, t) is just the transition probability ,,(%, x), which 
satisfies equation (4.36) with the boundary condition 


Pu(%%) =O for x = %, 4%. (4.107) 


Moreover, (4.107) holds for x) = %,, x2, since then the boundary is 
reached from the very outset, and all realizations are excluded from 
consideration. The transition probability also satisfies the 
Kolmogorov equation (4.39). 

Next, consider the mean first-passage time 


M(x) = re) = J W(b 0) dt, (4.108) 
regarded as a function of the initial value xp, where 
W(t, x0) = i) x Peig(%s Xo) dx» (4.109) 


To find an equation satisfied by M(x), we integrate the Kolmogorov 
equation (4.39) with respect to x from x, to x,. Using (4.109), we 
find that 


aw aw 1 aw 
Se = K,(x) S— + 1 Ki(%) 5 - 4.110 
at (Xo, Oxy 2 (Xo, x8 ¢ ) 


Then, integrating (4.110) with respect to ¢ from ty to ow, and 
taking into account (4.108), (4.101) and (4.102), we have 


dM 1 &M 
1 = Kil) Ge + 7 Kaleo) Ga (4.111) 
a) 


If the initial point x, lies at the boundary itself, then the boundary is 
“reached immediately,” and the mean first-passage time is zero. 
Thus, equation (4.111) satisfies the boundary condition 


M(x) = M(%) =0. (4.112) 


We are now in a position to write down the solution of (4.111) ina 
general form. This allows us to find the mean first-passage time 
(4.106) for an arbitrary initial distribution (4.97), without having 
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to solve the nonstationary Fokker-Planck equation (4.33). In fact, 
substituting the expression 


: w(x, t) = f * pul %q) W(X) dXg 
a ™ 
ipto (4.100) and (4.106), we obtain 


' 
i Te = f M (Xo) o(%0) 20 » (4.113) 
a 


after using (4.108) and (4.109), Equation (4.113) is the generaliza- 
tion of (4.108) to the case of an arbitrary intial distribution w(x). 
For example, suppose that K, = 0, Kz = const. Then, the solution 
df (4.111) is 


1 
M(x) = — K, xy + Cy + Cy, 
which becomes 
i 
M(x) = K (% — *1) (¥2 — %0)» (4.114) 


after using the boundary conditions (4.112). Of course, in a more 
general case, the result is more complicated, but it can always be 
expressed in terms of quadratures. Another approach to problems 
involving first-passage times is given in Vol. II (Chap. 1, Sec. 3, 
and Chap. 2). 


) 
f 


‘7. Replacement of an Actual Process by a Markov Process. 
‘ A Special Case 

; Any actual process encountered in radio engineering, unlike a 
process which is exactly Markovian, satisfies certain conditions 
involving its smoothness, differentiability, etc. Nevertheless, in 
gome cases we can treat actual processes as if they were processes 
without aftereffect. In this section, we examine under what 
conditions, and with what justification, this can be done. 
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Suppose that the process x(t) is due to the action of another 
process £(t) on a system whose behavior can be described by the 
first-order differential equation® 


& = «Fle, 9), (4.115) 


called the fluctuation equation. Here, ¢ is a small parameter, and 
F is a known function of the arguments x and £. We shall assume 
that the process é(t) describing the external noise perturbations is 
stationary and has a finite correlation time 7,,. 

We begin by considering the special case where the function F 
does not explicitly depend on x. Then, since F[£(t)] can be regarded 
as a new random function, there is no loss of generality in writing 


k= e&(t). (4.116) 


Choosing t = 0 as the initial time, we can write the solution of 
(4.116) in the form 


st) = 5) +e f ee) de", (4.117) 


If the initial value x0) = xy is not random, the cumulants 
k, of x(t) can be obtained by integrating the correlation functions 
R(t, «-, t.)¢ of the process €(t): 


h=e I if “ Rltyontedt, dt, (s>1). (4.118) 
0 0 


Using the cumulants &,, we can write the characteristic function 





es < (iu)! 
O(u, t) = exp >» a hf 4.119) 
of the random increment x(t) — x9. Then, taking the time derivative 


* More generally, F can also involve the time ¢ explicitly, in which case 
equation (4.115) becomes 
& = «Flx(O, £0, ¢). 
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of (4.119), we find that the characteristic function satisfies the 
differential equation 





_f< (iu)? 5 
@= [x =~ k,| 8. (4.120) 
Next, we find the corresponding differential equation for the 
probability density 
(s — yt) = 5 fe Olu, t) du. (4.121) 


Taking the time derivative of both sides of (4.121), and substituting 
(4.120) into the integrand in the right-hand side of the resulting 
equation, we obtain 





=5 ky 1 jut) * en iul2— 
t= St ag J (intermete Btu.) du. (4.122) 
Since 
xf (iu)* e-tult—%) O du = (- zy + f e-iule—t) @ du 
oy! 
= (a) 
it follows from (4.122) that 
(HD yg ow 
o= 3S re (4.123) 


Thus, we have obtained a stochastic equation of the type (4.20) 
for a process which is not Markovian. Even in the case of a 
stationary process £(t), the coefficients &, depend on the time. 
However, this dependence is appreciable only for time intervals of 
the order of the correlation time (¢ ~ 7,97), and the coefficients k, 
are practically constant for t>>7,,,. In fact, according to (4.118) 
and (2.2), we have 


a cf it 
By mets [oe [Ruth thy ona te-adedty «dt 
€ J, i: (t,t; edt; ... dts 


C) 
= €'s i} oie f RG (Typ over Tag ET y oe AT 
-t -t 
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If we assume that the integrals 


0 
fi Peeve tea de edree © >1) (4124) 
are absolutely convergent, it follows that 


: k, > &K, 13 too, (4.125) 
where 


0 
Ks (ef Ad(tgs on Tot) dry ou dry (8 > 1) (4.126) 


are the intensity coefficients of the process &(t). It can be shown that 
the K, are also given by the expressions 


0 0 
K,= J os J Ri ayy oo0y Tyg) Ory von rg 
(4.127) 
0 on ea, 
=s! 18 dr, J dry + J, Ry 45 6) Ts-1) FT 5-1» 


where the coefficient K, coincides with (2.8): 
K,=K= 2 J” Ar)dr. 
° 


According to (4.125), for sufficiently large time intervals, 
equation (4.123) takes the form 





Bs =F Pek, 22. (4.128) 


If the parameter ¢ is small, only the lower-order derivatives play an 
important role, and the higher-order derivatives are unimportant. 
If we omit terms corresponding to s > 2, equation (4.128) turns 
into the Fokker-Planck equation. 

Ofcourse, the fact that the probability density w obeys a 
stochastic equation does not mean that x(t) is a Markov process. 
For this to be the case, one would have to be able to express the 
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multidimensional distributions as products of the transition 
probabilities, after writing the transition probability itself in terms 
of the probability density (4.121), iie., 


PAX, Xo) = W(% — Xq, 7) 


If a multidimensional distribution factors into a product of such 
functions, thig means that the random increments 


X, = x(t,)— 20)  X, = #(t) — a(t),  (O< <KX<..), 
(4.129) 

corresponding to successive time intervals, are independent of each 

other. Therefore, the Markov condition is equivalent to the con- 

dition that the increments (4.129) be independent. In an actual 

case, however, where the correlation time is finite, there is correla- 

tion between the increments (4.129). 

For example, consider the increments 


X= x(n) 20) =e feat, 


XKedita asf edt, (4.130) 


corresponding to the adjacent intervals [0,74] and [r,, 7 + 72]- 
The covariance (or cross correlation) of X, and X, is given by 


tts 


Ty 0 T; 
K[X,, X,] = ¢ J . A(t, — ty) dty dt, = if it i A(te — ty) dt, dt. 


(4.131) 


ay 


If the integral 
f ty — th) dy dt, = f ” ahr) dr (4.132) 
oo ° 
is absolutely convergent, then, according to (4.131), we have 


KIX, X:] = [ AG) dr +, (4.133) 
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where D->0 as 11,7,->©. It is appropriate to define the corre- 
lation time by the formula 


th(r) dr 0 
a Ae = + 7) dr, (4.134) 
0 


which is somewhat different from (2.7), but of the same order of 
magnitude. Then, 
K[X,, X.] > 5 Kreo (4.135) 


aS 74,7, -> ©. 
On the other hand, the variance of the random increments 
X,and X, is much greater than (4.135). In fact, 


DX, =e [" [kets — ty) dty dty = 2¢ ” ny — 2) R(x) dr 
ovo 0 
= @K(r — ter) + Di, (4.136) 
DX, = 2¢ f "rg — 2) R(r) dr = OK (ry — Teme) + Des 
v 
where D,->0 as 7, >, and D,->0 as r,-» ©. It follows from 


(4.135) and (4.136) that for large 7, and 7., the correlation coefficient 
of the increments X, and X, is approximately 


pa Kn Xd _ 





it T 
ee 4.137 
o(X,) (Xo) 2V (41 — Teor) (72 ~ Teor) f } 


which is small when 
T1> Teor s Te > Teor + (4.138) 


Therefore, we can neglect cross correlations, if we only consider 
intervals with lengths much greater than the correlation time. The 
same is true of the higher-order correlations, the only difference 
being that 
Pompe (i nan f "dts f "Ra Tea) Orga (8 > 2) 
K, oO 0 0 

(4.139) 

should now be chosen as the correlation time. 
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Thus, for time intervals which are considerably greater than the 
correlation time, the increments (4.130) can be regarded as com- 
pletely independent, and the process x(t) can be regarded as a 
Markov process. This means that the process é(t) with correlation 
functions k,(t,, ... t,) can be replaced by a delta-correlated process 
whose correlation functions are 


K Sty — t,) «.8(t, — ty), (4.140) 


with the same intensity coefficients 
0 0 
K,= f 26; f Ri typ oeey Ty) Oty ons Ory (4.141) 
-0 /-@ 


as the actual process é(t). 


8. Replacement of an Actual Process by a Markov Process. 
The General Case 


We now consider the more general case where no special 
restrictions are imposed on the function appearing in the right- 
hand side of equation (4.115): 


& = eF[x, O)] = F(x, 1). (4.142) 


Then the derivation of the stochastic differential equation for the 
one-dimensional probability density is more complicated. Speci- 
fying the value 

(tg) = %o (4.143) 


at the initial time ty, we consider the increment 
x(t) — x9 = H(Xo) = H(xo, t, ty) (4.144) 


which now depends in an essential way on the initial value x). To 
find H(x9), we solve equation (4.142) with the initial condition 
(4.143). In fact, we look for H(xo) in the form of an expansion 


H (x9) = €Hy(%») + €?H4(xo) + «.. (4.145) 
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Substituting (4.145) into (4.142), we find that 
. <i OF Z 
cHy + @H, +... = €F (x) + € By M0) (eH, + @H, +...) 
OF 
ae 5 53) [eH, + 2H, +. PF +... 


Equating ternis of the same order in ¢, we obtain 


Fhy(xq) = F(a) 


Hela) = F(a) He) 


3 (4.146) 
Hoy) =F (9) Hage) + 5 $5 (0) HYCO) 5 
so that 
t 
Ay(x9) = I, F(X, t’) dt’, 
IF te 2 
Hee) = [dt Frnt) [Fein t”)de", (4.147) 


After we have expressed H(xo) in terms of F, to some desired 
accuracy, we consider the statistical characteristics of H(x,). The 
characteristic function of H(x,) can be written in the form 


certs = 1 SEM cry. (4.148) 


By evaluating the inversion integral, we find the one-dimensional 
probability density 


wax | m9) = ic f OH 1 + 5 CH¥(x)>] du. (4.149) 
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Interchanging the order of integration and summation in (4.149), 
we have . 


w(x | x9) = [! fs > 4 (- =) <H(x,)>| a f e-iule20) du 
= (4.150) 
= [1+ 2 (- a) Crew] ae — 0. 


Unlike (4.148), this last expression is of a somewhat formal 
character, since the sum of derivatives of the delta function does 
not converge in the ordinary sense. To arrive at expressions that 
converge in the ordinary sense, we have to integrate the product of 
(4.150) and a function f(x.) from some suitable class of functions, 
obtaining 


Jee) M00 dry = Fe) +S (~ G) Keren sea 
We note that here the function (x) is first multiplied by (H*(x)), 


and then the product is differentiated. 
It is now convenient to introduce the operator 


i ya (-2) aren, (4.151) 


where in each term, the operation of multiplication precedes that of 
differentiation. Using L, we can write (4.150) as 


w= (14+ L) d(x — x). (4.152) 
Differentiating (4.52) with respect to time, we find that 
wb = 18(x — %). (4.153) 
Using (4.152), we can write the last equation in the form 


@=L(1+Ly', (4.154) 
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¥ 


where the operator 
a+¢Ly's=1-L4+R—-Ly+.. (4.155) 


is the inverse of the operator 1 + L. 

Equation (4.154) is the stochastic equation we are looking 
for, and the operator L(1 + L)-! appearing in (4.154) is given in 
terms of the expansions (4.145), (4.151) and (4.155), which are all 
power series in the small parameter «, since H and hence L are of 
order e. Substituting these expansions into Z(1 + L)-* and grouping 
terms involving the same power of ¢, we obtain an expansion for 
LU. +L), whose first few terms will be explicitly calculated 
below. As the time interval ¢ — ¢, is increased, the convergence 
of the series (4.145), (4.151) and (4.155) becomes worse. However, 
the convergence of the series for L(1 + L)“! does not become worse; 
instead, this series converges to a limiting series which is indepen- 
dent of the initial time ¢. 

We now find the terms of the series for L(1 + L)-* which are of 
order ¢ and «2. Then, in all intermediate calculations, we need only 
retain those terms which are required for this purpose. According 
to (4.145), (4.151) and (4.155), we have 


L=(- &) <AY + (- zy <HH) + O(6) 
=(-2) cm, tem + (- 2) emp + 0@) 
and 
(+L =1-L+0@)=1~(-Z) ap +o 
=1-<(-Z) a + 0@)- 
Multiplying Land (1 + L)-?, we obtain 


La +Lyt= 2 td +e ay) H,> 
(4.156) 
-¢2 ay Za + Of). 
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In the last term of (5.156), we interchange the order of the operators 
ajax and <H,(x)>. In fact, using the identity 


Aya LE — 2 fry) Fee) —§ A? feu), 


we easily see that 


a¢H(x)> 
ax? 


Ha) 2 = 2 Grey - (4.157) 


and hence 
2 tod ao = & city ty — 2 (8) erty. 


Finally, taking (4.146) into account, we can write the operator 
(4.156) as 


ca $Ly) = 2 aH — 2x [et ml res & KF. H+ 0(8). 
(4.158) 


By neglecting terms of order e? and higher in (4.158), we find that 
equation (4.154) reduces to the Fokker-Planck equation (4.33), 
where 








OF (x) 
K,(x) = «(F()) + &K L Hyx)|, 
ms [ae | (4.159) 
Kx) = 2°K[ F(x), 4y(x)], 
or, because of (4.147), 
0 a 
Kye) = «Fay +e f 8 FO) |e] dr, 


K,(x) = 2¢ f KLF(x, t), F(x, ¢’)] dt! = 22 J i K[F(s), F,(x)] dr, 


where 
F(x) = F(x,t +7). 
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As t — ty -> ~, we see that 


Kx) > K(x), K(x) > M(x) + 4K), (4.160) 
where 
M(x) = «<F(x)> 


Ke) =2¢ [ KLF@) Fall dr, (4.161) 
kw) = 4 f) K([FO, Fy]. 


For time intervals ¢ — to, which are considerably greater than the 
correlation time 7,,, of the function F[x, &(t)], or of the function &(¢), 
equation (4.154) is practically the same as the Fokker-Planck 
equation 


w= 7 |[Me) ah GK] oo yp 13 [K(x) o()]. (4.162) 


By analogy with (4.134), it is appropriate to define the correlation 
time by the formula 


0 
fans ie fo le liste, Fy dr. (4.163) 


For time intervals t, — t), t, — t,,... which greatly exceed the 
correlation time (4.163), the multidimensional probability distri- 
butions factor into products of transition probabilities (4.149) 
satisfying equation (4.162). The reason for this is essentially the 
same as in the preceding sections, but the proof is more com- 
plicated. Then, the random process x(t) will be approximately 
Markovian, and, to this extent, all that has been said about 
Markov processes will apply to x(t). 
According to (4.21) and (4.16), the coefficient 


K(x) = M(x) + 4 K(x) (4-164) 
is the “average derivative” 


tim 29? = Cay = CeFIa(), LOD = Ky(n)- 
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It follows that if we wish to calculate the mean value of 
«F[x(t), €(t)], we have to allow for correlation between x(t) 
and &t). According to (4.161), as a result of this correlation, 
<eF [x(t), €(t)]> differs from the mean value 


M(x) = «<F[x, EN), (4.165) 


calculated without regard for correlation between x(t) and é(t), by 
the quantity? 


é [x oF] & + 0). (4.166) 


Similarly, to calculate the mean value of any other function 

G[x(t), €(#)] of the random process x(t) satisfying equation (4.142), 

subject to the condition x(t,) = x,, we have to allow for correlation 

between x(t) and £(t) by using the formula 

K 2920. F(x] de + 0(@, 
(4.167) 


<Glx(t ANd = Glew GOD + « 


where the first term in the right-hand side is the mean value 
calculated without regard for correlation between x(t) and ¢(t). 

Among the possible functions F(x, €), an important position is 
occupied by those which satisfy the relation 





oF oF, 
fx mF ,] ae =f «[F ee, (4.168) 
which can be regarded as a kind of time-symmetry condition (i.e., 
symmetry with respect to the time reflection t-> —t). If (4.168) 
holds, then K‘(x) can be obtained by differentiating K(x): 
i) = 22, (4168 


7 When calculating the averages in (4.165) and (4.166), the argument x is 
regarded as a fixed quantity, rather than as a function of time. 
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Then equation (4.162) can be written in the form 
o=~= 2 (mw) +4 7 x o («2 +t +2 ku). (4.170) 


If M and K are independent of time, (4.170) gives the stationary 
distribution 





Wax) = 7S exp {2 ie ae dy}, (4.171) 


which is equivalent to (4.49), because of (4.160) and (4.169), and is 
valid under the same assumptions. 
In particular, the relations (4.168) and (4.169) hold in the special 
case where 
2 = eF[x, &(t)] = f(*) + 8(*) &(t) i (4.172) 


here, f(x) and g(x) are known functions, while &(t) is a stationary 
random perturbation with zero mean value and intensity coefficient 
kK, Le; 


dy =0, 2 f dE) dr = K. (4.173) 
In this case, according to (4.161) 
M(x) =f(x), K(x) = ng%(x), (4.174) 


and the stationary distribution (4.171) becomes 





Cc 2 

W(x) = 77) exp \= ie a @ y| (4.175) 
We can also solve the inverse problem, which consists in finding 
a fluctuation equation (4.142) for a random process satisfying a 
given Fokker-Planck equation. Of course, in general this problem 
does not have a unique solution. However, the solution will be 
unique if we restrict ourselves to equations of the type (4.172), 
containing a Gaussian delta-correlated random process £,(t) with 

zero mean and unit intensity: 


<folt)) =0, — <bobor? = 8(r), (eK = I)- (4-176) 
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Then, using (4.164), (4.169) and (4.174), we find that 


ae) = VE), fe) = Ky) — , ee, 


(4.177) 
Therefore, the arbitrary Fokker-Planck equation (4.33) corresponds 
to the fluctuation equation 


#= Kye) - 4 Go) 


+ VKats) felt) (4.178) 
We can associate a fluctuation equation of the form (4.178) with a 
more complicated fluctuation equation (4.142) which has the 
same Fokker-Planck equation; two such equations are said to be 
(stochastically) equivalent. This method of simplifying the equation 
of a random process by replacing it by an equivalent equation turns 
out to be useful in solving certain problems, and wiil be used later. 

Finally, we consider briefly the error committed when we 
discard terms of order «? and higher in (4.154) and (4.158). 
Retaining all the terms would lead us to a stochastic equation (4.20), 
where each coefficient K,(x) can be expressed in terms of F as a 
power series in the parameter € (beginning with terms of order «°), 
whose terms can be found by applying the method used above. We 
are mainly interested in the limiting values of the coefficients 
K,(x) which come into play when the influence of the initial time ty 
vanishes, i.e., when t — ty S> teor- Thus, we go one step further 
in the direction of making formula (4.159) more precise, by retaining 
terms of order ¢? in our calculations and neglecting terms of higher 
orders. This leads® to the following formulas: 


Ky=M+ 4K 4 sl. eee 


ee ae Fe aeke 


oF ee 











=. FK [5—, 








8 The details are given in the Supplement, p. 126. 
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0 0 
K,=K+28f" f k [2 FF] dr do 


sa afl fan] + (Ee awy 





~ <F)K [ne al do, 


K, =3¢f . ii K[F, F,, F,] dr do, 
Gane: (4.179) 


Here M, K' and K denote the expressions (4.161) previously 

found, of orders € and e?. In the special case where the fluctuation 

equation has the form (4.172), the formulas (4.179) can be con- 
siderably simplified and reduce to 

ag a of 

eli aiat Seales leg 8 rag)? 


Ky = Kg? + age + xg? 5 (£) : (4.180) 
Ky = 1g, 


where 


m= 2) <e de, 
a ap. [xt £,, 6] dr do , 


ox ie Ir] CEE) dr. 


We now examine the relative size of the terms appearing in 
(4.179). ‘Taking account of (4.161), we can make the following 
order-of-magnitude estimates: 

oF 
4K ~ 8 Freer * 
(4.181) 
OF 5 


mar Toor $e (2y Freore 


K,-M-}K'~eS 
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Provided that 
OF OF \? 
(ar)? < (ze) » 
we can conclude from (4.181) that the terms in e? in the expansion of 
the coefficient K, are smaller by a factor «(@F/@x)r,,, than the 
term in ¢®. Similarly, we can convince ourselves that this factor 
describes the relative size of the correction K, — K, compared 
with K,. In fact, more generally, the corrections due to higher-order 
approximations to the coefficients K,, involving higher powers of 


the parameter e, have relative sizes described by an appropriate 
power of the ratio 


Teor OF 
whe ~ € Fy Teor - (4.182) 
Here, 
-1 
™%)~ Ie = (4.183) 


is a time constant, which can be called the relaxation time. A 
necessary condition for the effectiveness of the methods developed 
in this section is that the inequality 


OF 
€ Dy Teor <1 (4.184) 


should hold. 

We introduced the small parameter ¢ in order to make it clearer 
how to construct successive approximations in determining the 
coefficients of the stochastic equation. However, in all actual cases, 
« occurs in combination with the function F. In fact, in solving 
practical problems, one knows the whole right-hand side «-F = © 
of equation (4.115), and there is no need to introduce e at all. 
Thus, the quantity (2@/2x) 7,,, is actually the parameter which 
determines the convergence of the successive approximations. 
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9. Systems of Fluctuation Equations Involving Several Processes 
All the considerations of the preceding section can be imme- 
diately generalized to the case of a system of equations 
&, = €F (x, «05 Xp, t) = F(x, t), 
ee th hae (4.185) 
Xp = F(x, ..., Xp, t) = F(x, ¢), 
which define the random processes x,(t), .... xp(t). For fixed values 
of x, ..., %p, the expressions F(x, ¢), ..., F(x, t) are known random 
functions of time. Choosing the initial values 
X(to) = X19 (LE = Ley P) > (4.186) 
we write the solution x,(1 </ <p) of the system (4.185) as an 
expansion 
t 
a(t) — 0 = Here) + ef Faltorts) dt 
ro 
(4.187) 





2 pt OF ‘ 
+> é [ dt, _ (oe 41) f, Fen(%oy £2) tz + +. 
m=) eo rm ‘o 


The increments x, — x49, «.-; %» — %po have the joint characteristic 
function 


(exp D> u(x — *){) =1+ x iu,<H (x%9)> 
(4.188) 
+ 1% itt A i(%o) Hm(%o)> + +++ 


By analogy with (4.150), we can write the multidimensional 
probability density, which is the inverse Fourier transform of 
(4.188), in the form 
W(Xy — X yoy o00y Hp — Xyq) = (1 +L) b(x, — x49)... 5(%p — Xp), (4.189) 
where 

7 a 1 a a 
b= (— 5) Heer +3. (— ae,) (- Fag) Hue) re . 
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It follows at once from (4.189) that the probability density 
W(X, — X19) ++) Xp — %po) Satisfies the equation 


bo =L1 +Ly tw. (4.191) 


We must now calculate the operator L(1 + L)-!. Substituting 
(4.187) into (4.190), we obtain 


aoe > = if CF (x, ty)) at, 
$ef : at, { > (Fie) F(t) dt} (4.192) 


a ge ct et 
+ FZ aaaae |, |, Fue Pale to dts dla + OC), 
so that 
—— Le = {crusty +«f. Pie (x,t) F. (%t')) at’ 


(4.193) 
+ 4% sae, |, PHO False dt + 0). 


Just as before, the operator L(1 + L)-! differs from (4.193) only by 
having the correlation functions 


SF Fal KiFo Fel 
in place of the averages 
Fry), CPiFa) 
As a result, equation (4.191) becomes 
‘ se 
ee _ (fo +E PK Fat Fas ar) wo 


(4.194) 
- +See axa, if K[Fi, Fe] dr 0} , 


tet 
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except for terms of order <? and higher. Here, in calculating the 
averages <F;> and the correlation functions in the integrands, the 
arguments x,...,%, are regarded as fixed. For t — t > 7eor, the 
lower limit t, — t can be replaced by —o. 

We see that to within the given accuracy, the equations (4.185) 
are equivalent to the multidimensional Fokker-Planck equation 
(4.79). We now show that, conversely, every Fokker-Planck 
equation can be replaced by a system of fluctuation equations 


41 =f) + foal) fx(t) (= lyonp), (4.195) 


where the functions f,(x) and g;m(x) are suitably chosen. Here, 
€,(), --» €p(t) are independent, Gaussian, delta-correlated random 
functions with zero mean values and unit intensities: 


<> =0, — <rEme> = 8im8(r) - (4-196) 


Applying formula (4.194) to the equations (4.195), using (4.196), 
and comparing the result with (4.79), we obtain 


Kis) =f) +55 Fae Be 
, (4.197) 


Kin(%) = x 8%) Bil) + 


We can always go from the Fokker-Planck equation to the system 
(4.195), provided that the relations (4.197) can be solved for 
the functions g,,,(x) and f,(x). But this can always be done, as 
we now show. The matrix K =||K,,,{| is symmetric and 
positive definite (more precisely, nonnegative definite), and hence 
it follows from familiar results of linear algebra that there exists a 
ceal, symmetric, nonnegative definite matrix G = || 21, || which 
1s the square root of K, i.e., 


G=K, G@=K, DSaukim=Kim- (4.198) 
Es 
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In fact, because of the symmetry condition gj, = 2mj (4-198) is the 


same as the second of the equations (4.197). It follows that the 
functions gm, and hence the functions 


fie) = Kix) — ly ae Sim (4.199) 


an always be determined. 
To calculate || gim ||, we can use the orthogonal matrix 


J = || 44 || which reduces K to diagonal form, i.e., 
UKU- = || K%,,||  (K9>0). (4.200) 


In terms of U, we have 
G = U* || VK98,,|| U- (4.201) 
‘Using the u,,, we can write the equations (4.195) in the form 
Heh = Yah + VR), (4.202) 
“where they have been solved for the “‘fluctuations.” Here, 


u= x uty (= 1, P) 


ire a system of random functions just like the €(¢), and in fact 
satisfy the same conditions (4.196). 

In the diffusion approximation (which corresponds to neglecting 
ierms of orders ¢, ¢*,... in the stochastic equation), any system 
of equations (4.185) can be “‘simplified,” in the sense of being 
replaced by an equivalent simpler system (4.195) or (4.202). 


10. Gaussian Markov Processes 


Within the class of Markov processes, the simplest are those 
which are described by a Fokker-Planck equation (4.79) where the 
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coefficients K,,, are constants and the coefficients K,(x) are linear 
functions: 


Kix) = — D Bimm tO: (= bye) (4.203) 


According to (4.195) and (4.197), this Fokker-Planck equation is 
equivalent to the fluctuation equations 


dn + Bim = H(t), (4.204) 
where 


at) = b + DY) Simbm(t) (4.205) 


are Gaussian random processes such that 


mM = bh 
K[n1, Mr] = Kimd(r) - 


The system of linear equations (4.204) can always be solved, and 
as a result, x,(t), .... ¥p(t) can be expressed linearly in terms of the 
known random processes 7,(t), -... np(t). Therefore, the processes 
x,(t), -.-, ¥p(t) are also Gaussian, and their statistical characteristics 
can be easily determined. 

In order to calculate the cross-spectral densities of the processes 
21, +4) Xp» We replace the differentiation operator in (4.204) by 
iw and find the inverse of the matrix 


(4.206) 


I] Bim + £23 im || - 


Let A(iw) be the determinant of this matrix, and let 4,,,(éw) be the 
cofactor of the element in row / and column m, so that 








{1 Bion “+ i208 pq [= = | “at | ; (4.207) 
and 
4 => 4), (4.208) 


Aa) 
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Using (4.208), we find that the cross-spectral density of the 
frocesses x; and x, is 


4 iw - tw 
Slop m5 o] = D2 = oe) Slny mie]. (4.209) 


But because of (4.206), (2.54) and (2.12), we have 
SE, Imi O] = 2m + 4b ybp5(w) « 


In particular, setting j = k, we find that the spectral density of the 
process x,(t) [centered at its mean] is given by the expression 


S[x; — (xi e] = 2 | A(tw) |-* DY Anliw) 45,(@) Kim + (4.210) 
im 


Here, the numerator and denominator contain polynomials in 
4. A process with a spectral density of this kind is said to have 
a rational spectral density. 

Thus, we have shown that Gaussian Markov processes have 
rational spectral densities. The converse assertion is also true, i.e., 
every Gaussian process with a rational spectral density can be 
represented as a component of a multidimensional Markov process. 
This fact is important from a theoretical point of view, because the 
spectral density of any actual process can always be approximated 
to any desired accuracy by a rational function. Therefore, an 
:ctual Gaussian process which is non-Markovian can be represented 
approximately as the component of a multidimensional Markov 
-yrocess, and by increasing the number of components of the 
Markov process, the approximation can be made more accurate. 
This question will be discussed further at the end of Sec. 12. 

We now consider in more detail the simplest process x(t) with a 
tational spectral density. This process, which is simultaneously 
Stationary, Gaussian and Markovian, plays an important role in 
joise theory, and will be called the exponentially correlated process, 
1 designation whose meaning will be apparent from the formula 
derived below for the correlation function of x(t). According to 
4.204), for a one-dimensional process we have 


+pxe=n, (4.211) 
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where 
= b, K[y, 2] = K¥8(7) a 


In this case, formulas (4.208) and (4.210) become 


$ x= 





ptio’ 
2K UK 
TRt+io® ~ #+at 


To study what happens when the process x(t) is ‘switched on,” 
we impose the initial condition 


a(t) = 2%. (4.213) 
Then equation (4.211) has the solution 


Slx — <x)5 a] = (4.212) 


x(t) = xe Atte) f eA) (5) ds. (4.214) 
te 


Using (4.214), we find the following expressions for the mean value 
and the correlation function of x(t): 


¢ 5 b 
= — Alt—ty) — Alt— 8} lcs — 7) sue 
<x(t)> degen Rg 9 & ds 2+ (* aye tt) | 
ie (4.215) 
K[x(t,)s *(63)] = f° [eet teram (5, — 5) ds, dy 
tov te 
If to be explicit, we choose t, — t, > 0, then 


ty K 
K[x(t,), 2(¢,)] = K i} en Alt Hy-29) dy = R fem Altg ty) @ Alt tty 2t0)] 


In general, we have 


K[x(¢,), *(4)] = ¥ eT Pita“! — @ Alt thy“ 240)] (4.216) 


Ast — t, increases, the ‘‘stationary” correlation function 


Mr) = ORG), ot =e, Rey er*hl, (4.217) 
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corresponding to the spectral density (4.212), becomes “‘estab- 
jished.” Moreover, according to (4.216), after the process is 
‘switched on and while it is reaching equilibrium, its variance is 
given by the formula 


Dat) = o7[1 — R{t — t)]. (4.218) 


Since the process x(t) is Gaussian, just like 4(), the mean value 
(4.215) and variance (4.218) completely specify the one-dimensional 
probability density 
: = I [x — m — (x) — m) R}? 
W(x) = Prt, Xo) = TVR exp} — ee , 

(4.219) 
where 


R=Rt—t), m= 


The probability density w(x) corresponds to the condition (4.213), 
and hence is just the Markov transition probability (4.4), in terms 
of which the arbitrary multidimensional distribution (4.6) can be 
written if we know the initial distribution. 

The formulas (4.215) and (4.216) are special cases of the general 
formulas (3.26). Therefore, the distribution (4.219) gives the 
conditional density w[x(t) | x] for any stationary Gaussian process. 
However, (4.219) can be used to construct the multidimensional 
distributions only in the case of the exponentially correlated 
process, for which R(r) = e~4"!, 


11. Second-Order Fluctuation Equations. 
Solutions of Special Cases 
of the Two-Dimensional Fokker-Planck Equation 


In many problems, the noise process x(t) under consideration 
satisfies a second-order fluctuation equation 


# = Flx, %, &(t)], (4.220) 
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where &(t) is a random process with a small correlation time 7,o,. 
Introducing the notation 


t le 


Teor” Teor dE’ 


i= 








we can write (4.220) as 


dx 2 1 d& 
Ge = tee F (mT Ged) 
or as a system 
dx 
a Teor V > 
a (4.221) 


a = Toor F(x, ¥; )- 


The small correlation time 7,., in the right-hand side of (4.221) 
plays the role of the small parameter ¢ which appears in the right- 
hand side of (4.185) and allows us to go from the system of 
fluctuation equations (4.185) to the Fokker-Planck equation 
(4.194). In terms of the original symbols x and t = roof, the 
Fokker-Planck equation takes the form 


(x, 8) = -s20-2 \(<F> +f Ker] dr) w| 





ato, (4.222) 
+ oa \f KF drew}. 
We now analyze some special cases of (4.220). 
11.1. Let the fluctuation equation have the form 
wre + & — f(x) = E(t), (4.223) 


to which we can reduce the somewhat more general equation 


at + be = f(x) + a(8)- (4.224) 
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Moreover, let ¢(t) be a delta-correlated random process with zero 
mean, ice., 


<£)=0, Eb: = 18(r). 
‘Then if we write (4.223) as a systein 
yey, 

Wy —T +f) +e), 


the corresponding Fokker-Planck equation becomes 
; _ _ lp. aw wo 17a Ke Ow 
tis 9) = =F [ye + fe) Syl aed [50 + 3 ail , (4.225) 


from which we can derive the following results: 
1. It is easily verified that the stationary distribution corre- 
sponding to (4.225) is® 


2 = 
wu(%, 9) = Cexp{-242f° fe) ds}, (4.226) 
since when w has the form (4.226), each of the terms in brackets in 


(4.225) vanishes. It follows from (4.226) that the stationary 
distribution of x is 


w(x) = Cexp \2 f fea) dal, (4.227) 


which is independent of » and identical with the distribution of 
the one-dimensional Markov process described by the equation 


% = f(x) + &(t). 


In fact, this equation can be obtained either by setting » = 0 in 
(4.223) or by setting g = 1 in (4.172). Therefore, the stationary 
distribution (4.227) is of course the same as (4.175) when g = 1. 


* In this equation and the next two, C denotes the normalization constant, 
which is suitably chosen in each case. 
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The stationary distributions (4.226) and (4.227) exist subject to the 
same restrictions on the function f(x) as in the case of a one- 
dimensional Markov process. For a fluctuation equation of the 
form (4.224), the stationary solution (4.226) is replaced by 


woy(s, 8) = Comp] Sars 2 Ul" piayds + cepa] |, 4.228) 
where 


«=f x K[g(é), a(&)] ar - 


2. To find the nonstationary solution of equation (4.225), we 
write w(x, y) in the form of an expansion 


0(3 9) = D Pratt) Xnf ¥,(9) (4.229) 


in terms of the eigenfunctions X,,(x), Ya(y) which satisfy the 
equations 


5 Pal) — 2 f(a) Xm(2l] + AnXnle) =O, (4-230) 





5 RE) FVM + MYO) =0, 4.231) 





and the orthonormality conditions [cf. (4.56)] 
dx dy 
J Xn(a) Xue) ym Beer J Val) VOD YE = Bee 
Substituting (4.229) into (4.225), we obtain 
De PnnXm¥a = —1 Dy TanlXei¥¥n +L) Xm¥al — SMT mnXm¥a- 
Using the relation 


- Xny¥n +400) Xm¥n = dw | Xm dV, — Xm f Ya dy 
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which follows from (4.230) and (4.231), we can transform the basic 
equation into 
i 


Lebron + MTnn) XY = 1 ¥ TlMeXt [ Yn dy — de f Xe de Ya) 
: (4.232) 
. We now expand X;, and [X,dx in terms of the eigenfunctions X,, 
by writing 
Xile) = J Xm(=) ames 


(4.233) 
J Xels) dx = J) Xa) be» 
where the expansion coefficients 
dx 
me (bcp cae 
ank f mX X,’ 
(4.234) 


dx 
bk = | Xm(| X, de} = 
w= [Xm (f eae) 
obviously form matrices which are inverses of each other, i-e., 
D Amber = Snr 
ke 


We also introduce similar coefficients for the functions Y,(y) and 
JY.(y)dy. In fact, since equation (4.231) is the same as equation 
(4.68), it has the same eigenvalues A, = n and eigenfunctions 


Y,0) = Ne ae (V2) (yas) 


[cf. (4.69)]. It follows that 


Yip) = [2 VET Yen), 


as (4.236) 


J Ys) dy = 5 Ze Yaa) 


Substituting (4.233) and (4.236) into (4.232), replacing A, by n, and 
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equating coefficients of identical functions X,,Y,, we obtain the 
system of equations 


?Tan + 2T mn = Be fe Vat D GmeT ens 
re 
_ (4.237) 
2 
—p Re Va DZ bate nnn (m,n = 0, 1,2)..), 


which is equivalent to the two-dimensional Fokker-Planck equation. 

When the parameter yp is sufficiently small, we can solve (4.237) 
by a perturbation method, which goes as follows: Setting 2 = 0 in 
(4.237), we obtain 


PT mo = Ht ne Ds, mre T i » 
k 


where here (and from now on), p denotes the differentiation 
operator 0/8. But 7, can be found by setting 2 = | in (4.237): 


meee DD Tn — 7 rte 2 Yoh (4.238) 


Then, to find Tj, we set n = 2 in (4.237), obtaining 


Vip 2 
fees 2 ula — 353 @ YbAT as 


which we then substitute in the right-hand side of (4.238). This 
procedure leads to the following expansion in even powers of the 
small parameter pz: 


BAn 
wT, ~T ppp Tino 
2 4 


r (4.239) 
+ (+ 2p)? (2 + pp) Py An AeeAr Tio + O(n). 
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After deciding upon a given accuracy, we can discard all terms of 
higher orders, while simplifying the remaining terms somewhat. 
Thus, in the second approximation, we have 


[? + 1 eral Tmo = * Dy anedabusd Tv + Ou‘). 


Transforming the expression in brackets, we obtain 


P+ Am(L — pp)! = Pp + Am — HAmp + O(u4) 
= (1 — pn) [2 + Am (L — 2 7Am)-4] + O(eet) 
= (1 — p*A,,) [Pp + Am + 202] + O(ut). 


Therefore, to the same accuracy as (4.239), we can write 


Tro = — (Am + 2H) Tmo + pe D amedbirdrT ro + O(u'). (4.240) 
eat 


The equation (4.240) describes how the coefficients T,,9 vary in 
time. If » = 0, each coefficient T,,, decreases exponentially with 
‘{time constant” A;}, just as in the case of one-dimensional Markov 
processes (see Sec 4). Thus, if » = 0, a perturbation which has 
the form of an eigenfunction X,,(x) decays without changing its 
shape. However, if » + 0, the perturbation still falls off exponen- 
tially, but it has a somewhat different form. We now illustrate this 
situation by examining the form of the perturbation which falls off 
most slowly (with time dependence given by the factor et’), and 
which coincides with the first eigenfunction X,(x) when p = 0. 
Setting 3/2t = p, in (4.240), we find that 


Tmo = ae oy Amber Te + O(u'), (4.241) 


2 
Pi +A, + PM, a 


for m 4 1. Moreover, setting m = 1, we find that the constant p,, 
which determines the rate of decay of the perturbation, is equal to 


Pr = Ay WAR + WALD) amiAwia + Ont). (4.242) 
k 
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The one-dimensional probability density w(x) is given by the 
formula 


w(x) = Y Pao%Xn() ‘ (4.243) 


which can be obtained by integrating the expansion (4.229) with 
respect to y. Substituting (4.241) into (4.243), and retaining only 
terms of order »7, we find that the “nonstationary perturbation”’ 
which falls off most slowly while the stationary distribution is 
being established has the form 


m(s) = Tae [X(s) +H SE XaCa)] (4.244) 


and is proportional to the ‘‘modified eigenfunction” corresponding 
to X,(x). Similarly, we can find the shapes and time constants of the 
other modified eigenfunctions. 

x It should be noted that equation (4.240) is equivalent to a 
certain differential equation for the one-dimensional probability 
density w(x). To derive this equation, we multiply (4.240) by 
X,,Ax) and sum the result over m, as suggested by the form of 
(4.243). In doing this, we take into account the relations 


ZY Xml) anal = FS Xile) Ces 


mk 
D Xalt) AaCm = [2mm -4-4] DW Xnl2) Ce 


which follow from (4.233) and (4.230). Thus, the coefficients a,,, 
correspond to the differentiation operation, the coefficients b,, 
correspond to the integration operation, and the ,, correspond 
to the operation 

7) x 0 

a LO) — 5 a 
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‘Then a simple calculation leads to the differential equation 


is) = 2 [1 — 2 @)] [We oe) + £O], (4245) 





where f'(x) = Of(x)/dx. If this equation is regarded as an ordinary 
one-dimensional Fokker-Planck equation, the corresponding furc- 
tions K,(x) and K,(x) are just 


Ki) =f) — Y@s@) —H Ss"), 
Kz) = «fl —p¥'@)) 


Naturally, if higher-order terms in p» are taken into account, the 
corresponding equation will be more complicated. * 

3. If the parameter is not small, another method has to be used 
to find the solution of equation (4,223). Thus, we now consider 
the opposite extreme, i.e., the case where 7 is large. Writing 
1/u = ¢ and replacing ¢ by t/e, we can transform equation (4.223) 
into the new equation 

é B+ et — f(x) = Ve E(t). (4.246) 
j 

If ¢ is small, this equation describes the behavior of a system 
performing nonlinear oscillations under the influence of weak 
frictional forces and weak external fluctuations, It is convenient to 
introduce the “energy” 

E= $24 u(x), (4.247) 
where 
ie ef 
uz) = — [ fle) de 
° 7 
is the “potential function.” Multiplying (4.246) by %, we obtain 

B= —est 4+ Veet). (4.248) 
‘Thus, from (4.247) and (4.248), we obtain two fluctuation equations 
\ & = VIE — u(x)), 
EB = 28 — u(x)] + V2eE— wey] Ae). 


(4.249) 
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Then, in the usual way, we can associate (4.249) with the following 
Fokker-Planck equation for the probability density w(Z, x): 


tis, B) = — 2 [VIE — ww] + 2-5 [(B -u - 4) o] 
(4.250) 
+ xe 5 (E~u) a), 


where 


7 [i <e6o ar. 


If ¢ is small, the energy E is conserved during a large number of 
periods of the oscillations, and moreover, the time which (2) 
spends at the point x is inversely proportional to the velocity 
# = V2(E — u). Therefore, for a fixed value of the energy, the 
conditional probability density is 


le [E — u(x)}-1? for u(x) < E, 


w(x | E) = fw) >E, 4251) 


Normalizing this distribution, we can write the two-dimensional 
probability density in the form 


w(x, B) = (EB) w(x | B) = WBS (4.252) 


Here 


4 dx 
v(E) =5 Al san VES" (4.253) 
and the integration is over the region R(E) where u(x) < E. 

The distribution (4.252) can also be derived directly from 
equation (4.250). In fact, after a “‘quasi-equilibrium distribution” 
has been established, characterized by the fact that w is of order «, it 
follows from (4.250) that 


Z VE ae) u] = 0(6), 
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which when integrated gives 


v= iE). € 
Tea +: 


Thus, the deviation of w(E, x) from (4.252) is a quantity whose 
order of magnitude is «. 

Substituting (4.252) into (4.250) and integrating with respect to 
x, we obtain the one-dimensional Fokker-Planck equation 


wHE) = ‘= (Ser oe - ae] tes ele rE) wo] (4.254) 


for the energy, where 


#E) =f VE — u(x) de 
RUE) 


[cf. (4.253)]. This reduction of the two-dimensional Fokker-Planck 
equation to a one-dimensional equation considerably simplifies the 
problem of investigating nonstationary noise problems. Of course, 
if we take account of (4.252) and (4,247), the stationary solution of 
(4.254) reduces to the solution (4.228) found previously. 


11.2. Next, we consider the somewhat different equation 
| 


# + x, #) — f(z) = Vea). (4.255) 


In this case, the stationary solution of the corresponding Fokker- 
Planck equation can no longer be written so simply in terms of 
quadratures, as in the case of equation,(4.224). However, for small 
values of the parameter «, we can apply the method just described, 
which, in particular, allows us to find the stationary distribution. 

Thus, multiplying (4.255) by # and introducing the energy 
(4.247) instead of #, we obtain the equations 


SNAG Ps (4.256) 
—« VUE — u) h[x, V2(E — u)] + V2e(E — u) et). 
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The Fokker-Planck equation corresponding to (4.256) differs from 
(4.250) only by the form of the second term in the right-hand side, 
where instead of 2(E — u), we now have 


VUE —W) Mx, VE a). 


Substituting (4.252) into this Fokker-Planck equation and inte- 
grating with respect to x, we obtain the one-dimensional equation 


4B) = «3 [Sey — 5) 9) + Fam lgey el. 4259 


where 
1 
E cn am, I — 
a Rew i ees VUE —u)] dx. (4.258) 
From (4.258), we can find the stationary distribution in the usual 
way: 


w(B) = const 9(B)exp|~ 2 f° WEY apr}. (4259) 


Using the relation 


py 4) (4.260) 


to introduce the function F(E), we can write the two-dimensional 
distribution (4.252) in the form 


const 


2 
took B) = ames Om {- = F)| : (4.261) 


Going over to ¢ and bearing in mind that dx dE = % dx dx, we have 


t04(2, #) = Hew _ ar Fes ce wx)]| ; (4.262) 
where 


w= foo]-2F[hy + walle, 
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If we integrate (4.262) with respect to x, we find that the stationary 
distribution of the coordinate x is 


ois x exp{ — ip [5 yt us)]| dy. (4.263) 


When A(x, £) = 8%, the functions ¥(Z) and 9(E) are proportional, 
and then the distribution (4.262) coincides with (4.228). 


11.3, Finally, we consider the equation 
pit + [1 + pPA(x)] # —f() = &t), (4.264) 


which is more general than (4.223). The corresponding Fokker- 
Planck equation is 


pris,y) = —n fy 2 +40) 3 


: (4.265) 
+ wh) 50) + 5 00) + 58, 


where y = px. We shall assume that the parameter » is small, 
exploiting this fact to solve (4.265) by a method resembling that 
‘used on pp. 110-114. 

' First, we write the probability density w(x, y) in the form 


w(x, ¥) = Dy w(x) ¥a(y) , (4.266) 


where the Y,(y) are the eigenfunctions (4,235) of the equation 
‘4,231). Next, we substitute this expansion into (4.265), using the 
first of the equations (4.236) and the relations 


By On eye LVn+2¥nie— "Yn, 
(4.267) 


y¥, > — ia [Va +1 You + VA Yn), 


which follow from (4.231) and (4,236). Then, grouping coefficients 
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of identical eigenfunctions in (4.265), we obtain the system of 
equations 


(a+ atp + aint) oy = wal var Ste 
(4.268) 
ze if 2 Vain 4 kOe 
PNG PATO Oe 
—pvVn— 1 Vn hen.) 


where p = 9/8. This system can be solved by the method of 
successive approximations. If we choose the same accuracy as on 
pp. 112-113, then it is sufficient to retain only the first three 
equations, which correspond to n = 0, 1, 2 and have the form 


pap eal 2 
BPW = BAL By Me 


U2 a 2 L 
1 eV ete “ie D+ pp + ph Duy, 





(4.269) 
eae 282 Oe 
2 = PND 24 pp + 2uth dx? 
2 V2 weV2h 
+ ONT pp + yn OM TE ip ah 
a5 
where 
xk 0 
D=-f+55- 


Substituting the second of these equations into the first and 
third, and using the third equation as well, we obtain an equation 
which, to the required degree of accuracy, can be written as 


CZ) i 


= # _ pk & 4 
Peo = 5. TF pip Tain Do +o? a5 Dito Bq hit + O(n') . 


Transforming (1 + »2p + »7h)-} into 1 — pp — p*h, transposing 
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the term —27p(0/x) Dw to the left-hand side, and dividing both 
sides of the equation by 1 + #7(2/dx) D, we obtain 


(4.270) 


Then, using the form of the operator D, we combine the second 
and third terms in the right-hand side of (4.270): 


a 


Gxt 


o~ (50) = 5 [9 - Pad? 


are a a, 
2 lag lag) Peel 

Thus, equation (4,270) can be written in the form 
2 @ ‘ ow oe 
a) = 2 NU ~ wip’ — 28h) [fo + eal = 5 Fahne). (4.271) 
Here, we have replaced w(x) by wx), since w(x) is just the 
cne-dimensional probability density of the coordinate x, as can be 
verified by integrating the expansion (4.266) with respect to y. 
‘Thus, we have obtained a one-dimensional Fokker-Planck equation 
which is equivalent to the two-dimensional equation (4.265), to the 
cegree of accuracy chosen. In particular, using (4.271), we can find 
the stationary distribution 


w(x) = const - exp {2 * f(z) [L + p2h(z\) dz + p%h(x)!, (4.272) 
t “ 


which is a generalization of formula (4.227), If the fluctuation 
equation has thesform 

asi + bx) * = f(x) + &t), (4.273) 
then (4.272) becomes 


,(x) = const - exp {2 f I(z) W(x) dz + Ux). (4.274) 
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12. Transition from a Markov Process to a Smoothed Process 


The study of Markov processes is particularly appropriate, since 
effective mathematical methods are available for analyzing them. 
However, a certain care must be taken in replacing an actual 
process by a Markov process, since Markov processes have many 
special features, and, in particular, differ from the processes 
encountered in radio engineering by their lack of smoothness. 

For example, suppose that the process y(t) is described by the 
equation 


I —f0) = &(t)- (4.275) 


If the process é(f) has a small correlation time t,o, < t 9, where, 
according to (4.183), 
of \* 

~~ (ay) - 
then, with the accuracy discussed in Sec. 8, the probability density 
w(y) satisfies the Fokker-Planck equation (4.162). By writing 
(4.162), we replace the actual process y(t) by a Markov process 
x(t) which satisfies the equation 


& — f(x) = Mt), (4.276) 


where {(t) is a delta-correlated random process with the same 
intensity K as &(t). Since {(t) has infinite variance, it follows from 
(4.276) that the derivative x has infinite variance (i.e., Dz = ), 
which indicates that the function x(t) lacks smoothness and that its 
graph is “jagged.” However, the inductances and capacitances 
always present in the circuits used in radio engineering act to 
smooth the process x(#), so that to obtain a process like those 
encountered in practice, we have to subject x(t) or {(t) to smoothing 
with a time constant r,,, < ro. [In the case described by (4.275), 7,,, 
is of the order of the correlation time of the process £(t).] The 
smoothing operation considerably modifies the behavior of the 
process x(t) over time periods of the order 4t ~r,,, but the 
“large-scale” fluctuations (for which At ~ 7 S>tg_) remain 
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almost unchanged. Thus, the results obtained by applying the 
techniques of Markov process theory are valuable only to the 
extent to which they characterize just these “large-scale” fluc- 
tuations. 

To get an idea of how the operation of smoothing affects a 
Markov process, we consider a specific example of smoothing, 
obtained by setting 

it 
at) = f htm ue’ydt’. (4.277) 
This corresponds to assuming that ¢(t) in (4.275) is an exponentially 
correlated random process, satisfying the equation 
tome + & = X(t). (4.278) 
According to (4.275) and (4.278), we have the following second- 
order differential equation for the smoothed process y(t): 
Tom¥ + [1 — Tomi’ NI — fy) = Mt). (4.279) 
We arrive at approximately the same equation if in (4.276) we 
average x(t) instead of f(t), by setting 
ae . 
Ht) = fete ale") dt, (4.280) 
ie., 
Tan + Y= *- (4.281) 


Then (4.276) and (4.280) imply the equation 
Tom +I —SL(tem¥ + ¥) = Kt), 


which coincides with (4.279) if we expand f(t. + y) in a power 
series in +,, and neglect terms of second and higher orders in T4n- 

Equation (4.279) corresponds to a two-dimensional Markov 
process, and is a special case of equation (4.264), whose solution was 
studied in Sec. 11. Applying formula (4.272), we find that the 
stationary solution of (4.279) is 


wa(y) = const - exp | [" fle) dz — PY) — tm"). (4.282) 
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Equation (4,282) shows the extent to which the smoothing perturbs 
the original stationary distribution. 

If f is a linear function, i.e., if f(x) = —Ax, then a(t) is an 
exponentially correlated process, and 





hel) = ae 


(cf. entry 2 of Table 1, p. 25). The smoothed process y(t) has the 
spectral density 

S[xyo] 2K 
wre, +1 (ort, + 1)? +B) 





Sly] = (4.283) 


According to entry 7 of Table 1, y(#) has the correlation function 
As) = ii (1 — Be2,)-* [etl — Bre ttiitm], (4,284) 

where 

K 

= S(14+Brm). 

3g (1 + Brom) 

If Br,,, < 1, we can simplify (4.284) somewhat: 
R(t) = alee! — Brome! tem]. 


The smoothing operation just described is incomplete. In fact, 
(4.279) shows that the variance of the second derivative of y(t) is 
infinite. To eliminate this infinite variance, while remaining within 
the framework of Markov process theory, we would have to 
consider a fluctuation equation of at least the third order. However, 
any random process actually encountered in radio engineering is 
analytic, and all its derivatives are finite with probability 1. There- 
fore, we cannot describe an actual process within the framework of 
Markov process theory, and the more accurately we wish to 
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approximate such a process by a Markov process, the more com- 
ponents the latter must have and the higher the order of the 
corresponding fluctnation equation must be. 

- As an example, we consider the analytic random process which 
has the correlation function 


R(t) = oe PH 
and the spectral density 


Stéio] =F vacom. (4.285) 
(cf. entry 3 of Table 1). Equation (4.285) can be rewritten in the 
form 

eae 4c? = 
err PS[é; wv] = - 7. (4.286) 
If in (4.286) we make a power series expansion of e**/* and retain 
only the first 2 + 1 terms, then the process ¢(¢) is replaced by a 
Markov process €,(t) and (4.286) becomes 

(1+ 5 Pte toe im oe) S1&qi o] = Ao ve (4.287) 
In fact, we can represent the first factor in the left-hand side of 
(4.287) in the form P,(iw)P,(—iw), where P,(p) is a suitable 
polynomial, and we can represent the right-hand side of (4.287) as 

the spectral density of a delta-correlated process {(t) by writing 


3 S| [25 ow) or e oa 
1 
Then (4.286) becomes 
| Paw) P S[En5 w] = Slt, e], 


which is equivalent to the following differential equation of order 2: 


P, ($) é, = X(t). (4.288) 
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This implies that 


Ge, antg, 


bas GP ey “Got 


(i.e., the process é,(¢) and its first » — 1 derivatives) form the 
components of an n-dimensional Markov process. Moreover, the 
larger n, the less €,(t) differs from &(t). If n = 1, the process £,(¢) 
is an exponentially correlated Markov process satisfying the 
equation 


pith =U), 


and 


Ebi = 0 Vr eroll, 


The process €,(t), which is a closer approximation to £(t), has 
the correlation function 


bk: ox) = a 





w big a. 
ai irifcos br +5 sinb| >|], (4.289) 


where 








_ V24+1 a Vv2—1 a 
a=8 z+ «(Ob = 8B ye Re v24+1, 
(cf. entry 5 of Table 1), and satisfies the differential equation 


aight At ese tL 





wi 


Supplement 


x Since the calculations leading to the formulas (4.179) are rather 
complicated, we give some of the details here: It follows from 
(4.151) and (4.145) that 


LI -L4+R—.)=«4+0B4+68C+.., 
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where 
C= ~ 2H) + Stal) + HD) - a SCA 
- 2 [ato Zao + dog ao] bet ub 
+ Snny Zap - ae 2 diy Z Hy) 2H). (4.290) 


We write (4,290) in the form 


a tad 
c=-204 2G, —53%s, (4.291) 
where C,, C, and C; are functions which can be expressed in terms 
of derivatives and integrals of the function F appearing in (4.142). 
If we use the relations 


aH, 





BODE is 0 <i) = aCe ) <> 


2H 








2 ty Zap = 5 tp 





Zp & ap = Bo ap 
oH, OH, 
2 ase a Gat ) SAD » 
Btn 2 Hy = & ta) (Hd 
oH, a 
by -2@G H,) «Hy (a 
Ori a 18. 7] 
Za dy 2 dy = Bao cH" 
FZ fe( 2B) crayt + cat (24) cy] 


AIC (28 2} 


FA) HD, 
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which are consequences of the operator identity 
Ww e= 250) - LO. 
Substituting these relations into (4.290), we obtain 
C= tty ~ (FB) cay ~ (FB) cy — 5 (BE) cat 
+ (BBY cay + (Ah) (FB) cay, 
Cy = (HH, + (HyHy> — Hy) <H,) — (Hy) (Hy) — (3h <H}) 
— (i) diy — (EB) cry 4 2( 2B) cay 
+ <> (fh) <M), (4.292) 
adits — kenny <i — adh <I) + hy HD. 


Next, we substitute (4.146) and (4.147) into (4.292). Using (1.47), 
we easily see that 


Cy = FRM Ma = 3)" f° REP Fo Fe) dr do 
Similarly, the expression for C, is found to be 
ae oF H,, Hy] + le ey f Fey ae") 

El Eom dsne) 
~(F) (f fi ae Fee) [Fey ae") 
QUE oa (fnere), 
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We now divide the region of integration t' < t, t <t of the 
third and last terms into two parts, i.e., the region t’ > t’’ and 
the region t’ < t”. Then we use the identity 


<Enb> — <€) <nb> — End <> + E> > <> = KLE 0 ] + CKEE, C] - 


This leads to the formula 





Cy =3K [Fa HH] wits a SF, 
ees 


To obtain the final result, we need only make the change of 
variables o > +, 7—>o in the last integral. An expression for C, 
is found by the same kind of argument. The derivation of (4.179) 
is now straightforward. « 





oF | \dodr 











<F > dr do. 


CHAPTER 5 


Nonstationary Random Processes 


In radio engineering, one encounters many random processes 
other than the stationary processes considered in Chap. 2. In fact, 
as the subject of statistical radio engineering evolves both experi- 
méntally and theoretically, one has to deal increasingly often with 
the extensive class of nonstationary processes. In this chapter, we 
shall consider only a few very important types of nonstationary 
processes. 


1. Processes with Slow Nonstationary Changes 


This kind of nonstationary process is characterized by the fact 
that it behaves almost like a stationary process for long time inter- 
vals, which, however, do not too greatly exceed the correlation time. 
Although the mean value m,(t) and the correlation functions 
Ro{t,t + 7), R(t, t + 71,t + 7),... depend on the absolute time 
t (and not just on the time differences +, r,, 74, ...), they change only 
slightly during times of the order of the correlation time r,,,. As 
applied to the mean m, and the correlation function &,, this 
condition can be expressed analytically as follows: 


Om ok, 
PP woe him, Bre Eh. (6.1) 
For such a process, we can define a spectral density 


S[Es ot] =2 fo mts +2) eer dr, +(5.2) 
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which varies slowly with the time ¢. In the integrand of (5.2), we 
can replace m,(t,t-+ 7) by m(t—+,t) or by the symmetric 
expression m,(t — $7,t-+ $7). The spectral density (5.2) for a 
process with slow nonstationary changes plays the same role as the 
ordinary spectral density for a stationary process. 

The actual processes encountered in radio engineering always 
undergo slow nonstationary changes due to lack of stability of 
parameters of the equipment. These changes are unimportant if 
the time k,/(8k,,/8t) which it takes for the nonstationarity to 
manifest itself is much greater than the relevant time constants of 
the given problem. If this is not the case, one has to take the non- 
stationarity into account. 


2. Switched-On Processes 


In electrical equipment, noise processes do not become 
“established” at once; in fact, the influence of the initial conditions 
is “felt” for some time after the equipment is switched on. As time 
goes by, the influence of ‘the initial conditions ‘‘wears off,” and the 
“switched-on” process asymptotically approaches a stationary 
process. 

As an example, consider a stationary normal process é(), with 
mean value m and correlation function o*R(r). If the noise process 
&(t) has a known deterministic value £(0) = @ at the time t = 0, 
then, according to formula (3.26), the process is subsequently 
described by the conditional mean value 


m,(t) = m + (a — m)R(t) 
and the conditional correlation function 
A(t, + 7) = o°[Rir) — R(t) Rt + 7)). 


These quantities depend on t, and hence the process £(t) is non- 
stationary. However, after a time considerably greater than the 
correlation time has elapsed (i.e, when tS toon t+7> Teor), 
the functions R(t), R(t + 7) approach zero, and m,(z), a(t, t + 7) 
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Approach the quantities m, o®R(r) corresponding to a stationary 
process. 

In many cases, a noise process in a piece of electrical equipment 
is described by a differential equation, e.g., a first-order differential 
equation 


é=f(é) + &), (5.3) 


involving an external random perturbation ¢(t) which is stationary. 
The function f is usually such that as time increases, a stationary 
drocess is established in the system. However, in the beginning, a 
nonstationary transient process takes place, which can be studied by 
using the methods given in Chap. 4, Sec. 4. 


‘ 


3. Processes with Stationary Increments 


. When the function f(é) in equation (5.3) takes certain forms, the 
noise process é(t) never “turns into” a stationary process, no matter 
how much time ¢ has elapsed since the initial time. In particular, 
this is the case for the equation 


é= Ut), (5.4) 


which describes a process with stationary increments. Because of the 
stationarity of the function ¢(t), the increment 


totT 
Hto+ T) — Ht) =f" teyat 


during an interval of length T has the same statistical properties, 
regardless of which time ty is chosen. (In what follows, for brevity 
we set ty = 0 and é(t)) = 0.) If we subtract from the integral 


a(t) = [Mey ae (5.5) 


its own mean value, and then square and average the result, we 
obtain 


Dé?) = is icc —t,) dtd, = 2 [.@-akeeyar. (5.6) 


134 GENERAL THEORY OF RANDOM PROCESSES [cH. 5 


Differentiating (5.6) with respect to t, we obtain 
d U k 
De) = 2 f sho) a (5.7) 


Sometimes it is convenient to write (5.7) in the form 


DH) = 2 J” hele) dr 2" hee) dr 


or 
HM) = ESE — <0] —2f" ke) dr. (58) 


To determine the variance Dé(t) as a function of time by using 
(5.7) or (5.8), we have to integrate both sides of the equation with 
respect to t, using the initial condition 


Dé0) = 0. (5.9) 


Once we have found Dé(z), we can easily calculate the correlation 
function &,(t,, t,). In fact, if we substitute 


a= (ty) — Ei), b= E(t) — <E (te) 
into the identity 
ab = $ [a +  — (a — 5), 


and then average the result, we obtain 
RAty te) = 4 {Dé(t,) + Dé(t2) — D[E(t,) — €(¢2)]} « 
However, because of the stationarity of the increments, we have 


D[ét1) — €(t2)] = D&I t, — 21)» 


and hence 
Ae(ty te) = 3 {(DE(t,) + DE(t2) — DE(| t, — ty I} - (5.10) 


We now consider some examples. 
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Example 1. Let {(t) be a delta-correlated normal process with 
zero mean value. Substituting the correlation function k,(r) = K8(r) 
into (5.6), we obtain 
: Dé&(t) = Kt. (5.11) 
Therefore, the distribution of the increment of £ during time t is 
4 i ol _ 

’ “(6 = V2rKt oo | 2Kt . 
Because of the delta-function form of the correlation function 
kr), increments of the process ¢(¢) during nonoverlapping time 
intervals are independent. ‘Therefore, é(t) is a Markov process with 
transition probability 
1 (é, — £9 
7s = Abeesl 5.12 
At Oe OP IKe (5.12) 
Sometimes this kind of normal process with independent increments 


is called a Wiener process. According to (5.10), the increment (5.5) 
has the correlation function 





het) = Zl te — 14 — tl ]- (5.13) 


Example 2. Suppose now that the stationary process {(t) has 
the spectral density 

S[t; #] = awi-*, (5.14) 

where ‘ 

O<a<l. 


Then, applying the formula 


J * etme des = (ir) °T() 

0 

we find that 

Ret) = x Re f | metas = 5 608 le —a) 5] TQ —«)74 


, 


(5.15) 
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for 7 > 0, or 


aA uid 
hel 4 pane (5.16) 


where we have used the familiar formula 


v 
MM) = 
involving the gamma function. 

When + = 0, the expression (5.16) is not valid, since it can 
easily be seen that k,(7) takes arbitrarily large values near + = 0. 
In fact, including the value +s = 0 in (5.16) is like including a 
delta-function term of the form c8(r), where c is infinite; the 
resulting infinite variance Df = k,(0) is of the same charaeter as 
the infinite variance of delta-correlated noise, and is related to the 
presence of high-frequency spectral components in S[f; w]. 
However, in actual processes there are practically no spectral 
components with frequencies exceeding a certain upper cutoff 
frequency w,. Hence, the variance of an actual process is finite, 
and has the order of magnitude 


a wit 


MeO) a Danes” 


(5.17) 


As a — 1, the process £(t) turns into delta-correlated noise, while 
for « = 0, formula (5.15) gives 


1 
k()=-$G (7 >0). (5.18) 
Next, we calculate the variance of the integrated process é(t), 


using formula (5.8). Taking into account (5.14) and (5.16), we 
find that 


d a terl 
eo *3 Gre Danie. GP) 
Integrating (5.19) with the condition (5.9), we obtain 
a i= 
Dé?) 3 Tet lsin fra O<a<l). (5.20) 
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The case « = 0 requires special treatment. According to (5.8) and 
(5.18), we have 


d a 
Spee) =<. (5.21) 
The integral of (5.21) is divergent, but formal integration gives 
Dé(t) = S(Int — ino). (5.22) 


The infinite value In 0 is due to the presence of high frequencies. 
If we introduce an upper cutoff frequency w,, then the integration 
of (5.21) should be carried out from I/w, to t, rather than from 
0 to t, so that formula (5.22) is replaced by 


Dé(t) = S Inwt  (« =0). (5.23) 


From a knowledge of the variance Dé(t), we can find the 
correlation function of the increment of &#) during the time t. 
Using (5.10), we obtain 


_att+ty—|t,—t.|* 
Ret te) = 3 ne andes O<a<1) (5.24) 


and 


Adteste) = Zin Pl (@ = 0). (5.25) 





If we now fix the value of the difference t, — t; = 7 and let the 
time t = }(t, + t,) approach infinity, then, as can be seen from 
the expressions just obtained, the correlation function will not 
approach a finite limit, so that the “‘switched-on” process £(t) 
never achieves a stationary regime. However, under the condition 
+ <t, the expressions (5.24) and (5.25) can be simplified somewhat. 
By making Taylor’s series expansions with respect to r, we find 
that 


Cg cS eres O<ae<]), 


in(t +5) + in (t —$) =2Ine— Bs («= 0). 


Fo 
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Therefore, with an error of order 7*(7/t)?-*, we have 


2t% — 7% 


4T(@+ lsin fra (O<a<l), (5.26) 


k(t — 5, t+5)= 





A(t—5.t+5)= p10" a Se (5.27) 


T 


If the integrated process ¢(t) were stationary, then it would have 
the spectral density 


S[f wo] = 4 4 Sis ow) = ats ‘ (5.28) 


: 
corresponding to the correlation function 


re 


a 
~ 4 Ta +1) sin $a 





k(t) =C O<a<1), 

(5.29) 
kis) = C— aint («=0), 
where the constant C is arbitrary. It is true that for an ideal process 
such that the formula (5.28) holds over the whole frequency range 
right down to zero frequency, the constant C has to be infinite in 
order to satisfy the requirement that the correlation function 
k{z) be positive definite. However, for actual stationary processes, 
(5.28) holds only for frequencies w > w, which are not too small, 
ie., which exceed a lower cutoff frequency w, while for smaller 
frequencies w <w, S[é;] approaches a finite limit of order 
aj/w;**. Then, the constant C in (5.29) is finite and depends on 
w,, and (5.29) can be written in the form 


wy* — 7% 


i T(@ + 1) sin $a 
k(t) = ne (a= 
t 


Rr) = (0<a<]l), 


(5.30) 


These considerations show that for observation times t <1/w, 
a process é(t) with the spectral density (5.28) can be regarded as a 
nonstationary process with the correlation function (5.26) or 
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(5.27), while for observation times t > |/wy &(t) should be regarded 
as a stationary process with the correlation function (5.30). The 
difference between these two cases is tantamount to a difference 
in the interpretation of the constant C in (5.29). 


4. Periodic Nonstationary Processes 


According to (2.1), the statistical characteristics of a stationary 
process are invariant under arbitrary time shifts (corresponding to 
replacing t by t + a, where @ is arbitrary). On the other hand, the 
statistical characteristics of a periodic nonstationary process are 
only invariant under shifts by a multiple of a certain period Ty 
(corresponding to replacing t by t + a, where now a = nT, and 
n is an integer). As a result, the probability densities, moment 
functions and correlation functions of a periodic nonstationary 
process &(t) depend on the absolute time, as well as on the time 
differences, but the dependence on the absolute time is periodic. 
If we represent the process ¢(t) as a trigonometric series, its 
mean value and (second-order) correlation function can be written 
in the form 


o « 
mt) = > m,e7inUtlTo) — my + 2 Re > m,etntiTe) (5.31) 


nae nal 
R(t - = t+ 5) = hj(r) + 2Re Yraennwrr (5.32) 


where in general the coefficients m, and k,(r) are complex 
[m_, = mx, k_(7) = kq(r)]- 

If we shift é(t) by an amount a, where 0 < a < To, we obtain a 
random function é,(¢) = & (t + a), whose statistical properties are 
different from those of the original process é(t). In fact, in the 
general case, we have 


<Ea(t)> = m(t + @) # m(t), 
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Using the language of radio engineering, we can say that the 
processes ¢(t) and £,(t) have different phases. In some engineering 
systems, the phase of a periodic process is very important, and 
such systems are said to be coherent. In other systems, which 
are said to be incoherent, the phase of the process is unimportant, 
since the system does not respond to the phase, and instead carries 
out a time average. When dealing with incoherent sytems, we can 
replace a periodic nonstationary process by a stationary process, 
corresponding to a random phase “spread.” 

To illustrate this situation, we find the mean value # and the 
correlation function & of a random process é(t) = €,(t), whose 
phase @ is a random variable which is statistically independent of 
é(t) and has the uniform distribution 


ie r forO<a<T), 


0 otherwise. 
Averaging first with respect to é(t) and then with respect to a, we 
find that 
1 Te 
m= (nt +a)> = zl m(t + a) da, 
oro 
1 pT (5.33) 
Mig(ty, ty) == =f mt, + a, t, + a) da. 
o-o 
Setting 
m,{t, t') = m(t)m(t') + A(t, t') 
in (5.33), using the formulas (5.31) and (5.32), and then carrying 
out the indicated integrations, we find that 
hi = tm, 


5 
F(t, ) = > mg qe"h-iTe + Ret) — te). 
n 


ao 
Subtracting i? from m,(t,, t,), we find that the correlation function 
is 


2anr 


Ar) = 2! mn |* 008 + helt) « (5.34) 
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To calculate the higher-order moment functions and the prob- 
ability densities, we can carry out a similar phase averaging. In 
particular, the two-dimensional probability density is 


Eis fn te) = Ze lb nth Hasta +0) da. (5.35) 


As a result of this averaging, the only dependence which remains is 
that involving the time difference 7, so that the process é(t) is in 
fact stationary. 

The above formulas can also be applied to the case where the 
original process €(t) is periodic, but not random. For example, 
suppose that 


et) = Asinugt = 4 (elo! — ertenty (5.36) 
Then 
2: A 
aire m= 7 my =m, =..=0, kp =0, 
and formula (5.34) gives 


R(r) = 4 cos WoT. 


The function (5.36) corresponds to the two-dimensional prob- 
ability density 


w(t, £, t,t’) = 8(€ — Asin wot) 8(£’ — A sin wot’) . 


Substituting this expression into (5.35), we find the two-dimensional 
distribution of the stationary process €(t) = A sin (wot + ¢), where 
the random phase g = wea has a uniform distribution: 


HE, by 7) = a VB BABE, — £,coswgr — VA — isin wyr) 
+ 8(&, — &, cos wor + VA? — & sin wor] (5.37) 


= I [sin wor | 8(€2 + &2 — 26,2, c08 wyr — AP sin? wyr) . 


CHAPTER 6 


Systems of Random Points 
and Related Random Functions 


As radio communication, automatic control, telemetry and other 
fields of electrical engineering continue to evolve, extensive use is 
made of pulse signals, where the positions, durations and other 
pulse parameters are often random, with various kinds of corre- 
lation between different pulses. Thus, in order to design radio 
equipment in a rational way, we have to know the statistical 
characteristics of random processes which are due to the super- 
position of random pulses. In this chapter, we shall devote special 
attention to the problem of calculating the spectral density of pulse 
sequences. The theory of random points presented here can be 
applied not only to pulse signals, but also to other cases where a 
random process can be decomposed into discrete elements or 
events, e.g., in the statistical theory of many-particle interactions. 


1. Methods for Describing Systems of Random Points 


In many cases, one has to deal with a set of points which have 
random positions in a given space, in particular, on the time axis. 
Such points will simply be called random points. The times at which 


1 For an application of the theory of random points to a problem of radar 
detection, see R. L. Stratonovich, The conditional distribution of correlated 
random points and the use of correlations for optimum extraction of a pulse 
signal from noise, Izv. Akad. Nauk. SSSR, Otd. Tekh. Nauk, Energ. i 
Avtomat., no. 2, 148 (1961). 
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various random events terminate, the points of intersection of a 
random function with a given curve, the centers of gravity of 
particles in statistical physics, etc., are all examples of random 
points. Of course, by specifying the coordinates of the random 
points, we might convert this problem into one pertaining to a 
family of random variables; however, this way of doing things 
is inconvenient, since the number of random points is often a 
random variable itself, and when this is the case, concepts involving 
space and time relationships (e.g., the concept of stationarity) 
lose their intuitive meaning. 

In the special case where there are no correlations between the 
positions of the random points, the distribution of the points is 
described by the Poisson law. However, the theory presented in this 
chapter includes all possible kinds of correlation between the 
random points, and hence the formulas that we derive are valid in 
the general case. In what follows, we consider the case of identical 
random points in a one-dimensional space, which can be interpreted 
as the time axis, but all our concepts and formulas can be gene- 
ralized to the case of systems of points in a multidimensional 
space and to the case of points of several kinds. 

A system of random points is completely characterized by the 
sequence of distribution functions 


Ailtss Salty tes Salty ter ta)y os (6.1) 


where 2), ¢, ¢3,... lie in the domain of definition of the system of 
random points (ie., the set of possible positions of the points). 
Here, distribution function f(z), ..., t,) gives the probability 


aP = [f,(ty ..., t.) + O(4)]4,... 4, 
that at least one random point falls in each of the intervals 
th<t<t44),.,t,<t<t,44, (4, <4,..,4,< 4). (6.2) 


It is assumed that the values ¢,,...,¢,d0 not coincide (t, # ¢, if 
t %j)and that 4 is so small that the domain's (6.2) do not over- 
lap. For random points characterized by continuous statistical 
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properties, the distribution functions (6.1) are continuous and 
the values which they take when arguments coincide, e.g., when 
t, = tj, are conveniently defined by the corresponding limits, e.g., 
as t; > t,. A desirable feature of these distribution functions is 
that they characterize local statistical properties at the positions 
corresponding to their arguments, without reference to statistical 
properties at other points in space. 

We now choose an interval0 < ¢ < Tin the domain of definition 
of the system of random points, and we consider the generating 
functional 


Lrto(o] = (TL0 + ol), 63) 


where is the number of random points lying in the interval 
0<2< T,and4,,..., t, are their position coordinates, The class of 
functions v(2) is restricted by certain requirements which depend 
on the nature of the random points and will not be discussed here. 
Then different systems of random points are characterized by 
different generating functionals, as illustrated by the following 
simple systems. 

Example 1. Suppose we know that a fixed number N of 
random points falls in the interval 0 <<? < 7, where T, > T. 
Moreover, let each point take any position in 0 <? < T, with 
equal probability, and let this position be statistically independent 
of the positions of the other points. Then the position of each point 
is described by the probability density 


w(t) = ford <tST, (l<j<N), 
and therefore 
deo att [apat 
+a altzf wd. 


Moreover, since the positions of the points are statistically 
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independent, the average of the product IJ[1 + o(¢;)] is just 
the product of the averages <1 + v(z;)>, so that 


Lele) = [1 + Poe 


Example 2. If in Example |, we increase the interval length 
T, and the number of points N in proportion, so that their ratio 
N/T, = B, which equals the average density of points, remains 
constant, then in the limit we have 


Lrle(e)] =i [1 +f ao atl” = exp jaf’ aeart. 


"This generating functional corresponds to a “Poisson system of 
points”, which will be studied in Sec. 3. Examples of more 
complicated generating functionals will be given in Sec. 4. 


In the case of a system of random points with continuous 
statistical properties, the generating functional is related to the 
distribution functions by the formula 


2. es 1 i 
Lefty] = 1 + py = ik o i} Pata os £3) ly) on W(t) dy on 
i (6.4) 


For other systems of random points, the distribution functions may 
have delta-function singularities at points where arguments 
coincide; then, formula (6.4) can be used to define the distribution 
functions in terms of the generating functional (6.3). 

The gencrating functional completely characterizes a system of 
random points, and in this respect, it plays a role similar to that of 
the characteristic functional (1.60) in the theory of random 
functions. In particular, if [¢’, ¢’’] is a subinterval of the interval 
[0, 7], then the generating functional L,,-[v] can be obtained by 
setting v(t) equal to zero outside the interval [z’, ¢’'], just as in the 
case of the characteristic functional. 
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Together with the distribution functions f,, it is convenient to 
introduce the correlation functions (of the system of random points) 
Billy), Balti» ta)» Ba(Las tas ta)s ove (6.5) 

defined by the formula 


(6.6) 
= exp (Sar Dy Eel sees Lg) Sq ere Zul y 


which holds for arbitrary 2, 2,,... and arbitrary ¢,, ¢,,... in the 
domain of definition of the system of random points. The corre- 
lation functions (6.5) bear the same relation to the distribution 
functions (6.1) as the correlation functions (1.64) bear to the 
moment functions (1.63).? Thus, applying the formulas (1.69) to 
f, and g,, we obtain 


Sulfa) = ail4y) » 
Solty, te) == Salta, te) + gilts) Silte)» (6.7) 
Sa(tas tor ts) = Bata» tay ts) + 3{ga(ts) Bolter ta)}s + Bulls) B1(te) B1(ls)> 


where the symbol {...}, has the meaning explained on p. 19. 
A system of random points is said to be stationary if the distri- 
bution functions describing it depend only on the time differences: 


Lofty tay ovry te) = fille = ty, wy ts — ty). 


The most commonly encountered case is that of ergodic systems of 
random points. These are stationary systems characterized by the 
fact that the correlations between the events consisting of points 
falling at various different time instants vanish as the time instants 


2 Cf. formulas (1.66) and (1.67). 
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are moved apart. In other words, the correlation functions of an 
ergodic system go to zero, i.e., 


Bdlyy ot) +O, (6.8) 


provided that at least one difference ¢, — t, of the arguments is 
increased without limit ( | t, — t; | > ©). This does not apply to the 
distribution functions, which under the same conditions reduce to 
products of lower-order distribution functions (just as in the case 
of ordinary moment functions, in the absence of statistical 
dependence). 

Choosing the points 7,, ¢,, ... in (6.6) closer and closer together, 
and suitably adjusting the quantities z,, z,,..., we can transform 
(6.4) into 


Lyfo(t)] = exp (Ss J i a edt saey te) Ot) ve O(t,) dt. dt, (6.9) 


[cf. the derivation of the characteristic functional (1.68)]. From this 
it is clear that the correlation functions can be defined as functional 
derivatives of the logarithm of the generating functional, i.e., 





— S Indole) 
Eslty, ony ts) = 3e(t)) .. 50(t,) es ac (6.10) 
while on the other hand, 
_ _SLrf00)] 
Ekta) = Ba oe BCE loca et) 





Certain quantities and functions characterizing systems of 
random points can be expressed simply in terms of the generating 
function. For example, let 2 be the (random) number of points 
falling in the interval [0, 7]. Then, the characteristic function of n is 


ce) = Lyfe — 1]. (6.12) 
The probability that no points at all fall in [0, 7] is just 
P(n = 0) =L,{—1]. (6.13) 
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Next, we present without proof? some formulas involving the 
conditional distribution functions p,(t,; T), po(ty, t23 T), .. Here, the 
function p,(t), ..» ts; I) gives the probability 


= [pty w+ tes T) + O(4)] 4). 4s (v.14) 


that just one random point falls in each of the intervals (6.2) and 
that n = 5, i.e, precisely s points fall in the whole interval [0, 7]. 
Unlike the ‘‘unconditional” distribution functions, the conditional 
distribution functions are no longer local statistical characteristics, 
since their values at the times 4, ...,¢, depend on the interval 
[0, 7] in question. In fact, the conditional distribution functions 
are related to the unconditional distribution functions by the 
formulas 


Paltyy on tes T) = Soe jr. aL Searltar or tenn) Utygy oo gyre (6-15) 


Dividing (6.15) by s!, the number of permutations of the points 
falling in [0, 7] (the points can be distributed in various ways 
among the elementary intervals [?,,t, + 4], .... [#5 t, + 4te]), 
and then integrating with respect to the variables t,,..., ts, we 
obtain the probability that precisely s points fall in [0, T]: 


Lt pk 
Pr=j=P,=5 if ia J 4 Paltiv oy tT) dty a dt, (6.16) 
The meaning of the normalized conditional probability density 


a(Lyp vey bg3 TO 
wo tyy yt) = Plaats) (s=1,2,..) (6.17) 
is quite close to that of en ordinary probability density. 
Solving the inverse problem, we can find the distribution 
functions /,, f,, ... from a knowledge of the distribution functions 


3 See P. I. Kuznetsov and R. L. Stratonovich, On the mathematical theory 
of correlated random points, Izv. Akad. Nauk SSSR, Ser. Mat., 20, 167 (1956). 
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(6.17) and the probabilities P,, P,,... that various numbers of 
points fall in [0, 7]. The resulting formulas 


2. ! T/T 

Sty wes te) = > eo Pay J, - iF Were(lay oor Cope) Besar oe Tore 
‘=O ? 
" (6.18) 


are the inverses of (6.15). If the number of points in n is not 
random, then (6.18) goes over into the expression 


! T /T 
Saltys or ts) = ca J, a J, Wy( ty oy ta) Utyr 1... dt, for s e 6) 
° 6. 
Sty 0 ts) = 90 for son. 


2. Random Functions 
Constructed from Systems of Random Points 


Using a given system of random points, we can construct random 
functions in various ways. For example, in many cases we are 
concerned with a random function defined as a sum of the form 


az) = 226 ts), (6.20) 


where the ¢; are random points, Q(z, ¢’) is some known function, and 
z denotes some set of arguments (which in particular might be the 
time 2). Among all functions of the form (6.20), a special role is 
played by the random function 


&t) = Dat— 4), (6.21) 
i 
which will be called the ‘‘random density function.” The more 


general random function (6.20) can be expressed as a linear 
transformation of the random density function (6.21): 


nla) = J Ole, #1) Ele’) ae’. (6.22) 


Therefore, to find the statistical characteristics of the random 
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function (6.20), we should first find the corresponding characteristics 
of the random density function. 

Just like any random function, (6.21) is completely described by 
its characteristic functional (1.60). If we take account of the 
properties of the delta function, we can write the characteristic 
functional as 


{ue} = (exp fi [” ae) e(eyae|) = (exp {eX mea} 


or 
@7{u(t)] = en es) : (6.23) 


Here, as in (6.3), 2 is the number of random points falling in the 
interval [0, T], and ¢,,...,¢, are the coordinates of the points. 
Comparing (6.23) with the formula (6.3) which defines the genera- 
ting functional, we find the formula relating the two functionals: 


6,{u(t)) = Lele — 1]. (6.24) 


Thus, if we know the generating functional, we can determine the 
characteristic functional, and hence the other characteristics of 
(2). In particular, we can determine the correlation functions of 
(2), which can be expressed in terms of the correlation functions 
(6.5) of the system of random points. The corresponding formulas 
can be obtained either by functional differentiation with respect to 
u(t), followed by setting u(t) = 0, or by using the expansions 
(1.68) and (6.9). For example,, substituting (1.68) and (6.9) into 
(6.24), taking the logarithm, and retaining only linear and quadratic 
terms in the u(t;), we find that 


fT -42 pT pT 
af ala) mle dy ot ff dallas) mn) ate) da deg + 
= xc) [emt — 1] dey 


1 Toe s ial ty) 
+5), J, Bally» te) [etl? — 1] fet? — 1] dey dty + 
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or 
apt a2 pT pT 
if. Pale) ult) dy + JJ. Rats te) ma) tn) + 0 
opt (20 °T 
=F failed aed dt +> faults) we) at (6.25) 


2 /T tT 
yf) J) eat 6) okt) te) dey dy + 
Then; equating the linear and quadratic terms separately, we obtain 
A(t) = gilts)» 
Rafty, te) = Br(ts) (ty — ta) + Salta te) - 


Similarly, writing more terms in (6.25), we can derive formulas for 
the higher-order correlation functions, e.g., 


(6.26) 


Baty, toy ts) = B(t, — te) 3(t, — t5) By(ty) 
+ H(t, — ty) Balt» ta)}s + Salta» fa» ts) » 
Reg(tyy tay tay ty) = B(ty — te) B(ty — ty) (ty — 4) y(t) (6.27) 
+ 4{8(ty — te) 5(ty — ty) Ba(ty ta)}e 
s+ 3{8(t, — t2) 8(ty — ta) Baty ta)}s 
+ 6{8(t) — £2) galtr» tas ta)}s + Balla» tay ts» ty)» 


corns 





Here, as on p. 19, the symbol {...}, denotes the symmetrizing 
operation. 

It is not hard to see that substitution of the expansions (1.66) and 
(6.4) into (6.24) gives the same formulas for the relations between 
the moment functions and the distribution functions: 

mi) = f(t)» 
meth, ty) = Wty — te) Alt) + faltys 42)» (6.28) 
tig (ty, tos ts) = S(t, — ta) B(ty — f5) Salt) 
+ 3{8(ty — te) falta, ta)}e + Salta» tar ts) » 
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From a knowledge of the correlation (or moment) functions of the 
random density function, we can find the correlation (or moment) 
functions of any other random process of the form (6.20), by repeat- 
edly applying the linear transformation (6.22). It will be observed 
that we are dealing with a completely symmetric situation: On the 
one hand, the relation between the g, and the f, is the same as that 
between the k, and the m,; on the other hand, the relation between 
the g, and the &, is the same as that between the f, and the m,. 
Moreover, there is a deep similarity between the mathematical 
apparatus of the theory of random functions and that of the theory 
of random points; in particular, this leads to the appearance of 
symmetric sums of the same kind in both theories. 

As already remarked, the results given here can be generalized to 
the case of a system of random points in a multidimensional space, 
by simply replacing ¢ by the coordinates 2, ...,z, of a random 
point in the space. For example, the multidimensional generaliza- 
tion of (6.26) is 


RyZp 1 Zp) = Bil2p «0 Zp)» 


Real Zyy cory B yh By coy BH) = By(Fpy ey Zp) B(y — 2%) «+ HZ, — 2) (6.29) 


A Bol B yp ones Zs Zp voy Zs) 


3. Poisson Systems of Random Points. Shot Noise 


A system of random points is said to be a Potsson system if 
there are no correlations (of any order) between the points, i.e., if 
the occurrence of one or several points (in certain places) does not 
change the probabilities of occurrence of the other points. In this 
case, all the correlation functions of the system of points vanish 
except g,(t,), Le., 


Sty tz) =9, Salty ter ts) = 9,... (6.30) 
‘According to (6.9), the generating functional then has the form 


P af 
Lekoe)] = exp } f ax(e) 0) at (6.31) 
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If we expand the exponential in series and compare it with the 
expansion (6.4), we easily see that the distribution functions factor 
into products of one-dimensional functions 


Satay over te) = Balls) «+ Bate) « (6.32) 
Using (6.15), (6.16) and (6.32), we find that 


Pat [[,exteyat]"exp | — J, exten ar| (6 =0,1,2,..). (6.33) 


The distribution (6.33) is called a Potsson distribution, and is often 
used to define Poisson systems of points. In the stationary case, the 
average density of the points is constant, and (6.33) becomes 


p, = Gt) par, (6.34) 


If we have a large number of independent events which take 
place (in time) with a finite average density, then the occurrence 
times of the events have a Poisson distribution. For example, 
consider the equilibrium current flow in a diode. When a single 
electron is emitted by the cathode and reaches the anode, it 
creates an anode current pulse i(¢ — ¢,), where the position of the 
pulse on the time axis is determined by #,, the random time at 
which the electron is emitted. Moreover 


f it —t)dt =e, (6.35) 


where e is the charge of the electron. Dealing similarly with the 
other electrons, we obtain a set of random emission times 


wilicp thy tap 


which correspond to a system of random points lying on the time 
axis. We shall assume that different electrons are emitted by 
the cathode independently of each other, so that the emission times 
have a Poisson distribution. In fact, there could only be dependence 
between electrons emitted almost simultaneously by neighboring 
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parts of the cathode, but this dependence can be neglected, if we 
assume that the dimensions of the cathode appreciably exceed the 
possible “‘correlation distance.” 

In terms of the average anode current <J,>, the average number 
of electrons which are emitted per unit time is 


_ ay 
s Ween 


e 


(6.36) 


and this parameter completely determines the system of random 
points. The anode current 


L(t) = x i(t — t) (6.37) 


‘s a superposition of separate pulses, and hence is a random 
function of the form (6.20), constructed from a system of random 
points. We now use the formulas of the preceding section to find 
the statistical characteristics of J,(t). 

First we assume that the finite size of the transit time from 
cathode to anode can be neglected. If we assume that this time is 
infinitely small, then each separate pulse can be regarded as a 
delta function, i.e., 


it — 4) = @(t— 4), 
and then the current (6.37) has the form 


I(t) = e D) 8(t — t) = e€(t), (6.38) 


where é(t) is given by (6.21). To find the correlation function 
of the current, we set g. = 0 in formula (6.26), obtaining 


her) = Felt, 0-47) = g8(7), (6.39) 
which implies 
Ka, Tor] = eTa5(7) 5 (6.40) 
according to (6.38) and (6.36). 
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The Fourier transform of (6.40) gives the spectral density of the 
“fluctuating component” J, — <J,) of the anode current: 


Sa — ads o] = eS[E — (£5 @] = 2eI> - (6.41) 


According to (6.41), which is known as Schottky’s formula, the 
spectral density of the anode current fluctuations is proportional 
to the charge of the electron. This is natural, since the fluctuations 
are caused by the fact that electrical charge is discrete (hence the 
term “shot noise”). 

Taking account of (6.27) and the fact that the correlations (6.30) 
vanish, we easily see that the higher-order correlation functions 
also have “delta-function shape”: 


K[La(ta)s oer Lo(te)] = €%¢5(t1 — fa) --» 5(ty — fy) 
= e118 ty — ty). (ty — t) - 


However, at high frequencies, corresponding to oscillations 
whose periods are comparable to the transit time, we have to take 
account of the duration and shape of the individual pulses, by 
replacing the delta function by some other function Git — ¢,), 
which is determined by the detailed motion of the electron. If the 
jth pulse is 


(6.42) 


ut — t,) = eG(t — 4), (6.43) 


then the sum of all the pulses is 
I(t) =e f Ge — 07) ey a", (6.44) 


This, together with formula (6.39) for the correlation function, 
implies that 
K[Ia, far] = & [ Ge — 9 Gt +7—Jards 
(6.48) 
= ele) f G(r + 0) Glo) do. 


Using (6.44), we can easily find the relation between the spectral 


sec. 3] SYSTEMS OF RANDOM POINTS 157 


density of the current fluctuations and the spectral density of the 
random density function; the formula relating them is just 


Sle — Clays w] = & | Fliw) P STE — (65), (6.46) 


where 
Fliw) = f e-“tG(r) dr. (6.47) 
Moreover, it follows from (6.39) that 
S[E — <>; @] = 2a, (6.48) 
and hence 
Sia — (la); w] = 2eCI,> | F(tw) |. (6.49) 


This formula differs from (6.41) in that it contains the extra factor 
| F@w) |?, which might be called the “frequency depression 
factor,” for a reason that will soon be apparent. It is easily seen 
from (6.35) and (6.43) that 


FQ) = [ Ge) dr = 1. (6.50) 


Therefore, the formulas (6.41) and (6.48) coincide at zero fre- 
quency. At other frequencies, | F(éw)| <1 and (6.49) gives a 
smaller value than (6.41) for the spectral density of the shot noise. 
If the electrons undergo uniform acceleration in the cathode- 
anode space, which is possible if the electrodes are planar and the 
space-charge density is small, then a simple calculation’ shows that 


Fin) = Gag ll — (1 tie) er, 


8 
ET 


where T is the transit time. 


(6.51) 


0 ee + 507} ~ cos wT — wT sin wT), 





4 We use the fact that if the initial velocity of the electron can be neglected, 
then T = dV/2m/eV and i(t) = (e/d)*(V/m) t = 2et/T*, where d is the 
spacing and V the voltage difference between the electrodes, and m is the 
mass of the electron. 
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x So far, we have assumed that all the electrons give rise to 
pulses of the same form. However, taking into account the effect 
of the longitudinal component »; of the initial velocity of the jth 
electron, we should write the jth pulse as 


i(t — t) = eG — 4, 4), (6.52) 


where the pulse shape now depends on v,. Adding up the pulses and 
introducing the two-dimensional random density function 


&t, 2) = > 8(¢ — 4) (v — a), (6.53) 
7 
we find that 


I(t) =e J i G(t — t', v) &(t', v) dt! dv. (6.54) 


Here, G(z, v) is a completely known function of two variables, 
determined by the detailed motion of the electrons. Thus, we 
are now dealing with a random system of points in the (#, v) plane. 
Assuming as before that the “‘initial data” of the different electrons 
are independent, we can again regard the points as forming a 
Poisson system. According to (6.29), in the two-dimensional] case 
we have 

KUE(é, 0), €¢ + 7, 0] = alt, 9 8) So 0} (6.55) 


instead of (6.39), and 
yy = eff G(t — t', 0) g,(t’, v) dt’ do =e fat) dv, (6.56) 
since the density g,(¢, v) = g,(v) is independent of the time and 
J G(r, v) dr = 1 


for any v. Letting w(v) be the normalized probability density of the 
initial velocity, we can write g,(v) in the form 


fa? 


é 





82) = w(v) - (6.57) 
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Then it follows from (6.54), (6.55) and (6.57) that 


KUlo, Loe] = 6 [ [ Gt — 01,0) Gt +7 — 2,0) exo) dt’ do 
(6.58) 
= eI,) f f Glo, v) G(r + 0, 0) w(v) do dv. x 


4, Systems of Random Points Defined by the First Two Moments 


In the theory of random functions, one sometimes considers 
only second-order correlations, i.e., only the first two moments of 
the random function. Moreover, the formulas (6.26) and their 
unverses 


a(t) = <6), 
Baty, te) = <E(t1) Elte)> — <E(ta)> <Elte)> — <E(4a)> S(t, — 42) 
show that the first two moments of a random function like (6.21) or 
(6.22), which is constructed from random points, are uniquely 
related to the first two distribution functions of the system of 
points. Therefore, in a theory involving only the first two moments, 
we can regard the functions g,, 2, or f;,f, as known. In general, 
however, the first two distribution functions do not completely 
characterize the system of random points, and hence, to describe a 
system of random points more completely, it may be necessary to 
consider higher-order moments. Nevertheless, in some special 
cases this is not necessary, and the system of random points, and 
also random functions constructed from it, are completely deter- 
mined by the first two moments. We now consider two such cases. 


(6.59) 


41, Systems of points correlated in pairs. Given a system 
of random points, we say that the points are correlated in pairs 
if all the correlation functions except g, and g, vanish, i.e., if 


a ea 
Then the generating functional (6.9) has the form 


T I T -T ‘ ; ; 
Lrlo(e)] = exp |f atedoerat +5 ff edt evoceyaleyaar'|. (6.60) 
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4.2, Systems of nonapproaching points. Suppose that all the 
correlation functions can be expressed in terms of two functions 
f(t) and R(t, t’) by the formulas 


Balti ses, t,) = (-1)"4 (a as DiAttD we Alte) {R(te, ty) as R(t ty)}s 
(6.61) 
where, as before, {...}, denotes the symmetrizing operation. Then, 
according to (6.9), the generating functional associated with these 
correlation functions is 
(i In [1 + f.TR(, t°) ft’) H(2') a7] 
° FL RG, Alt’) a(t’) at" 


Here we have used the formula 


S(t) oft) dt) . 
(6.62) 


Lr[ut)] = exp 





Si (part =F +), 
aS 
in carrying out the summation. 
The meaning of the function R(t, ?') ts revealed by setting 
s = 2 in (6.61); this gives 


Salta te) = —filts) falte) R(ty to) 
or 


Solty» te) = filty) Ata) TL — Rty t2)] - (6.63) 


It is appropriate to call R(t, t') the correlation coefficient, since 
(6.63) shows that the distribution function f, vanishes when 
R = 1. Moreover, when R w J, (6.62) becomes 


Po 
Lr{oe)] = 1 + f fl) 0) at, 


which, according to (6.4), means that all the distribution functions 
vanish, except the first. If r,,, characterizes how fast the correlation 
falls off, so that R(t, t’) ~ | for small time separations ¢ — t' < 740, 
then it is clear that our random points cannot lie near each other, 
ice., the intervals between random points cannot be much less than 
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Teor. Therefore, the random points with correlation functions 
(6.61) are called a system of nonapproaching points. In particular, 
using the expression (6.62), we can find the probability that no 
points at all will fall in the interval [0, 7]. According to (6.13), 
we have 


T £08 de 
Pin = 0} = exp | f PML fe AEA: fit) dt}. (6.64) 
° SF RY, ty f(t’) at 

1 In many problems, in order to obtain concrete results, it is 
expedient to substitute systems of nonapproaching points, of the 
type just described, for the more complicated systems of random 
points actually encountered. For example, formulas from the 
theory of systems of nonapproaching points will be used in Vol. II, 
Chap. 3, Sec. 2, to calculate the distribution of the durations of 
up-crossings (peaks) of fluctuation noise. 


j 5. Spectral Densities of Various Pulse Sequences 
j 

In this section, we shall calculate the spectral densities of various 
Kinds of pulse sequences. We begin by considering a sequence of 
pulses of “‘standard form,” described by the function G(z). In 
general, the positions of the different pulses along the time axis are 
random, so that the pulse sequence has the form 


at) = x Gt —t), (6.65) 


where the ¢, are random points. We can regard (6.65) as a linear 
transformation of the random density function (6.21), ie., 


ne) = f ie Gt — tet) dt’, (6.66) 


or in spectral form, 
Ne = Fliw) &., (6.67) 
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where 


Fliw) = f io evo G(x) dr, 


— 
Te = J oie de, 
t, = re f ~ evtt £(1) dt. 


We shall assume that the pulse sequence (6.65) is a stationary 
random process, which means that the system of random points 
giving the pulse positions is also stationary. Then it follows from 
(6.66) of (6.67) that 

<ny = F(0) <é), 


5 (6.68) 
Sys o] = | FGw) |? STE; o] « 


But according to (2.9), (2.12) and the second of the equations 
(6.26) or (6.28), the spectral density S[é; w] can easily be expressed 
in terms of the functions g, and g,, i.e., 


SE; wo] = 2g, + 2y(w) + 4agi&(w) , 


(6.69) 
STE — £5] = 2) +4), 
where 
y(a) = ee ef" gol) dr. (6.70) 
Substituting (6.69) into (6.68), we obtain 
S[n; @] = 4aF*(0) 235 2| F(iw) |? [yw ; 
[95] = 4a F (0) g78(w) + 21 Fléw) P LY) + ar] Cor 


Sly — <n>3 eo] = 2| FG) Pf) +41] - 


In the special case where the function G(r) is a rectangular 
pulse of fixed height G, and width 7, (see Figure 1), so that 


G, for O<7<%, 


G(x) = 0 for r<O0Orr>%, cm 


skc. 5] SYSTEMS OF RANDOM POINTS 163 
we have 
eer ae 
Flia) = [2 [1 —e"], (6.73) 
and then (6.68) and (6.71) become 
, <a> = Goraf» 
8G? 


ww 


(6.74) 





Sly — nyse] = SP sin? S* Pw) + al - 

_ Thus, we see that there is no particular difficulty in finding the 
spectral density of a pulse sequence when we kno w the first two 
functions g, and gz. However, if g, and gg are not known, we have 
to calculate the spectral density of the pulse sequence by a different 
method, and then use (6.69) and (6.70) to find g,, y and g, instead. 
We now consider an example of this type. 

_ Let the random points z; be such that the distances between 
neighboring points are independent identically distributed random 
yariables, with a given probability distribution w(f). We choose 
: = Qas the time origin, and we renumber the random points with 
positive coordinates t, > 0 in such a way that ¢, has the smallest 
soordinate. Thus, we have 


ho—bm=h, —t = bw, 


or 
: hebth h=pthate.. §F=H4tGte thre. 
: (6.75) 
as in Figure |. 
n 

: Sia To To= const 

G 
Of ta t 


Fig. 1. A pulse sequence constructed from the random points (6.75), 
where each pulse has height G, and width 7. 


164 GENERAL THEORY OF RANDOM PROCESSES [cH. 6 
To calculate the spectral density of the random function (6.21), 


we use formula (2.27), which requires that we pass to the limit 
e— 0 in the expression 


<4 —4]f) 


UE A= fem eae — emp per pertem +.) 676) 


where 


is the Laplace transform of the random function £(2). It follows that 


o wo 
. 2 — g-(P+P*)ty UC pee tC — POC tet Cg) 

IME ame loris D 2 » 677) 
where the asterisk denotes the complex conjugate. Next, we divide 
the terms in this double sum into two groups: The first group 
contains the terms for which j > k, and we write j — k = /; the 
second group contains the remaining terms, for which} < k, and we 
write k —j = 1. Then (6.77) becomes 


2 


J 
<LIés p> = etme | CPA DN Chet lg) PMC perbooet Cage) 
kel 1=0 
(6.78) 
a wo 
+> EPP G Hoe AGO Cg at eee toed | 
== 
To calculate the mean value of (6.78), we use the independence 
of the different random variables £,, from which it follows that 
every term of the double sum can be written as a product of 
characteristic functions corresponding to the distribution w(¢), 
taken with different arguments. Thus, writing 


Olu) = fet a0) dl = cet (6.79) 


(recall that £ can take only positive values), we have 


Key = Op), 


. (6.80) 
<e?"t = @lip*) = O*(ip). 
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Therefore, the mean value of (6.78) takes the form 


<ILEE: A] 1) = <ert49%) |S) ONG + I*) OMG) 
ken be) 

(6.81) 

+d Op + io) omip)| . 

1th 
Summing the series in (6.81) [which is easily done, since they are 
geometric series], multiplying by 2e, and then setting p = $¢€ — tw, 
we obtain 


€ z 2" 
({e[e5 —][) 
_ Sete) Olie) 1 4 8*[w + (é€/2)] 
1— Oe) (1 — O[w + Ged) ° 1 —O*lw + (e/[D]\ 
Finally, according to (2.27), to find the spectral density we need 
only pass to the limit « + 0. If w 4 0, (w) # 1, we immediately 
find that 


(6.82) 


aes 1 BX(w _ 2 1-16)|? 
: S[E; wo) = Ww \q= Oo) or O*(w) <> 11 — @(w) eee 
fince by (1.22), Bis “ 
SD 


. 
. For the frequency range lying near the origin w = 0, the 
situation is more complicated, and a separate argument is needed. In 
this case, we use the first few terms of the expansion 


oles ar efer jo rofies S)]. 50 
which gives 


1 + O*[w + (ée/2)] 
1-6 +O] | 1— Oo + Ged) 


- Slept aay |! + + + F)] 685 


= Sara ll +++ F)]- 
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We now use the relation 


€ 
Go pat Oe ee (686) 
which is easily verified directly by integrating both sides. Then, 
taking account of (6.82), (6.85) and (6.86), we see that the spectral 
density S[£; w] has a delta-function singularity at the origin. Adding 
this singularity to formula (6.83), which is valid for w 4 0, we 


obtain 
a) 2 1H 1O(w) PF | 4n8(u) 
Sl = TT ee) te (8) 


The presence of the delta-function singularity 





4n8(w) 
oy 
in (6.87) is due to the fact that the mean value <> is nonzero. In 


fact, <€> is just the reciprocal.of the mean distance <{> between 
the random points, i.e., 





=>. (6.88) 


If we center the random function é(t) about its mean value, then, 
according to (2.12) and (6.87), 


1-(6e)P 
Sé- sol = eA (6.89) 
It is easily verified that (6.89) can also be written in the form 
SIE - so] = BRET GS. (6.90) 


From a knowledge of the spectral an of the random function 
é(t), we can immediately calculate the spectral density of the 
pulse sequence (6.65). Thus, from (6.90) and the second of the 
formulas (6.68), we obtain 


Sly — (adie) = Zi Fe) P Rei 3. (691) 
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In the special case of identical strictly rectangular pulses of height 
G, and width 79, it follows from (6.73) that 


a _ int 72 1 — | Ow) |? 
Sly — <1 e] = 5a. sin? —° Tl = Qa) F° (6.92) 
These formulas allow us to determine the Fourier transform y(w) of 
the correlation function g,(r) of the system of random points under 
consideration. Comparing (6.69) and (6.90) and using (6.88), we 
find that 


= 2 Re 8)_ 
He) = TB Re Tey * (6.93) 


‘Then, taking the inverse Fourier transform of (6.70), we find that 
O() ane O(w) 


1 Or 
Bx) = mb ie e Res = Gl) =D ie T= 6(a) cos wr dw. 
: (6.94) 





| Using the formulas just derived, we can find the connection 
sbetween the characteristic function @(w) and the Laplace transform 
‘L[R(r); p] of the correlation function k,(r) = A(r). Differentiating 


i both sides of (2.14) with respect to 7, we find that 


oo -% im etoriw STE — (£5 w] dw. (6.95) 


‘Then, we substitute (6.90) into (6.95), obtaining 





dH) AF cimig 44 Oe) 
d= mG. te Ow) (6.96) 
cy Tek La) 9 


Hap fe ot Teele 


The second term in the right-hand side can be written in the form 


1 ig Bw) , 
aol. 6? tay tt NG} de (w’ = —w), 
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which is just the negative of the first term in the right-hand side, 
except that 7 has been replaced by —r. It follows from (6.79) that 


OCip) = [ert e(t) a, 


i.e., @(ip) is the Laplace transform of the function w(£). Moreover, 
@(ip) is analytic for Re p > 0, and the integral 


coo} 100 
m0) = ef eFO(u) do = ts [et OLip) dp (p = —iw) 


is just the inverse Laplace transform. 
Now consider the function 


1 + O(%p) 
6G)" (6.97) 


which is also analytic for Re p > 0, provided that { is not a 
deterministic quantity.§ Clearly, (6.97) is the Laplace transform of 
the function 


1 ooey Lt OC) g te 1+ 9) 
Fri PT Gp? a ae |) Te ey 


2nit 


which vanishes for r <Q. Therefore, the first term in the right- 
hand side of (6.96) vanishes for r < 0 and the second term vanishes 
ifr > 0, so that (6.96) can be written in the form 


|), ‘| = _p_ 1+ Op) 
A 2D 1 — O(p) 





for r > 0. It follows from 


Rr) = — "oar = [$a - [Sar 
that 
Limes] = 5 he [Fa] —2 [Fso]]. (698) 


tif ¢ = c with probability 1, where c is a constant, then (6.97) will have 
poles on the imaginary axis, i.e., for Re p = 0. 
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and hence (6.97) implies 


yb 14+ 8) 1 
Mik) 2 = 3255 T— 6G) — OF i629) 


A similar relation can also be written for the function (6.94): 








.p] = te) 
Med) #1 = 3 1 tia) ~ KEP * 
Next, we consider a sequence of pulses which are not identical. 
Specifically, let the sequence consist of strictly rectangular pulses, 
all of unit height, whose lengths vary from pulse to pulse, so that 
both the initial points and the end points of the pulses form systems 
of random points (see Figure 2). Thus, the random points on the 
time axis are now of two kinds. Actually, the problem could be 
reduced to the study of a system of random points of one kind, 
which lie in a plane rather than on a line. However, we shall not 
give the general theory here, but instead confine ourselves to the 
special problem just described. 


n 


(6.100) 





oto tar fh 


t, t 


Fig. 2. A sequence of pulses with random positions and random lengths, 


Thus, let the initial points t, and the points t; of the pulses be 
numbered in order of increasing time: 


we Sti <t <t<.. (6.101) 
Then, the pulse sequence can be written as 


at) = Ye na(t — 4)» (6.102) 
a 
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where 
_ 1 for O<t<r, 
PO lo tae te0 wad eS OF 


Instead of the function (6.102), it is more convenient to consider 
its derivative 





a(t) = &(t) = 2 se t) — &t —t)), (6.103) 


which generalizes the random density function (6.21) to the case 
of nonidentical points. If we wish to find the spectral density of the 
stationary pulse sequence (6.102), we have to use the fact that the 
spectral densities of &(t) and 7(t) are related by the formula 


Sts) = 4 S16 «) (6.104) 
or 


Shy — se] = J ste — yo - (6.105) 


We shall only consider the case where the following two conditions 
are met: 


- 1. The pulse lengths 
B—ha=p, (6.106) 


are independent, identically distributed random variables, with 
characteristic function 


9,(u) = <etry (6.107) 
for any j; 
2. The intervals between pulses 
t,t) =o, (6.108) 


are independent, identically distributed random variables, with 
characteristic function 


O,(u) = <e*ry (6.109) 
for any j. 
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Just as before, we use formula (2.27) to calculate the spectral 
density S[é; w]. Of all the initial times t,, let tg have the smallest 
positive value. Then the Laplace transform of the function (6.103) 
can be written in the form 


L[é; p] = e7-?4e — e- PH 4 Ph — en Ph — (6.110) 


This is the appropriate formula for the case where the origin t = 0 
lies in an interval between two pulses. In the opposite case, where 
the origin t = 0 is covered by a pulse, we have to write 


L[é; p) = —e-Pte 4 e-P — cP 4 oP — (6.111) 


There are definite probabilities P, and P, for each of the two 
relations (6.110) and (6.111) to hold; for the moment, we shall only 
consider (6.110). Taking (6.106) and (6.108) into account, we 
can write (6,110) in the form 


LUE; p) = er Po] — e- P01 4 em Per Pa — e-Pri-Pm—PPy 4] 


= eP D {exp [—plp, + 6, +» +p; + 9,)) (6.112) 


j=0 


— exp [—plor + 01 +... + 95 + pina] - 


To calculate < | L[é; p}"| *>, we multiply (6.110) or (6.112) by its 
complex conjugate. This introduces the double sum 


oO oo 
* 
yD 44, 
p=0 k=O 
where 
A, = e7?s — e- Phin, 


which we then divide into three separate sums, corresponding to 
j=hj>kandj <k. Thus, we write 


LE: AP = Tyas =D,+D,+E.. 613) 
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where 
> es Pr 
Dae = LAA; 2. = 2 Aat = 22 Arndt, >= De 


In more detailed form, we have 


Do = etre DY {exp [—Plpr + 01 + + oy + oI 
j= 
— exp [—Pp(pr + 1 +. +95 + psa] 
x {exp [—p*() + a +. +7 + o)] 


— exp [—P*(p, + 91 +. +o + pra)» 
(6.114) 


and also 


> = en (Dt pte 
Me 


x DD exp [—Plor + + Pegs + Gest) ~ p*(er +o tee + on)] 


kmO bes] 


— exp [—plpy + -- + pest + est) — P*(P1 + + oe + Ped] 
— exp [—pler + --- + onan + Pasta) — P*(y + + Pe + %)) 


+ exp [—plpr + --- + S42 + Pestsa) — P*(Oy + + Oe + Prod] 
(6.115) 


Averaging the last two expressions, we obtain products of the 
characteristic functions (6.107) and (6.109), since the random 
variables p; and o; are independent. Thus, (6.114) leads to the 
expression 


(2) = Seite OX(ip + ip*) OXip + ip*) 
x [1 — 0,(&) — 6,(@*) + O,(ip + #*)]- 
Summing the geometric series in the right-hand side and setting 


D+ p* = «, we obtain 


(Su) = TP eraereyig ll + 8H — Op) — efor. 
3 (6.116) 
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Similarly, averaging (6.115) leads to the expression 


(Z,) = BF [0% Ont) O49) 6460) 
— GF*Y(e) OMe) OF-Aip) OLip) — O(c) Oe) BY) Xp) 
+ OF (ie) Ofc) OX) OX]. 


Factoring out 9%(e)@k(ie){(ip)Ox(7p), and summing the geometric 
series in k and J, we obtain 





Ken#ty 9, (ip) Oa(tP) 
(2.) = 1=Byis BY) T— Oy OM) 


x [1 - 3S — ein) + 60). 


According to (2.27) and (6.113), the spectral density we are 
looking for can be found from the formula 


5 Slée] = tig ,) + lig (X,) + lig (2, ) 
(6.118) 
es him <X,) +2Re lim <%,) ¥ 


after setting ‘ 


(6.117) 





€ ‘ 
Bg See 


If w 40, 6,()@(w) # 1, the passage to the limit in (6.118) is 
trivial. In fact, since 


O,(ie) = 1 — Kp) + FX p® +.., 
O,(ie) = 1 — eKo) + $eXKo> +... 
we have 


€ 1 
T— @(te) 6,4) yey ne (6.119) 
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Therefore, it follows from (6.116) and (6.117) that 


2 1 
him <2) “Qe [2 — O(w) — 6F()] 


2 
= Oro Re [I — 6,(0)], (6.120} 


, (w) B,(w 
(3) = GyeeyT— eet Ba) P-aqy- 8) 


A special argument shows that the relations (6.120) remain the 
same in the range w ~ 0, 0,(w)@,(w) Fe 1. Substituting (6.120) into 
(6.118), we obtain the formula 





1 ore. Re LL = 84) (1 = 6.) 
Siiel= Gey Tea | OD 
In the above derivation, we used the expressions which are valid 
for the case where the time origin t = 0 lies in an interval between 
two pulses. In the opposite case, where the time origin is covered 
by a pulse, we have to use (6.111) instead of (6.110), making all 
appropriate subsequent changes. Thus, to obtain the final result, 
we have to average two different expressions of the form (6.118), 
where the first comes from (6.110) and occurs with probability P,, 
while the second comes from (6.111) and occurs with probability P,. 
However, it is easily seen that replacing (6.110) by (6.111) does not 
change the expressions obtained when we pass to the limit « > 0. 
Therefore, the need for the additional averaging disappears, and 
(6.121) is actually the final result. 
To find the spectral density of the pulse sequence (6.102), we use 
formula (6.104), oe 


2 [1 — §()] [1 — 6,(~)] 
= ts x . . 

nel = 3 Coy 1 eo). ) (6-122) 
There remains a certain arbitrariness in the definition of the value of 
S[n; w] for w ~ 0, An analysis of the expression (6.121) reveals the 

following behavior for small frequencies 
Re Lt = Sx) Ll = 62(e)] _ w? <e)*De + <0)*Dp 

1 — 6,(~) 6,(#) 2 [Xp + <o>F 








(6.123) 
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Therefore, (6.122) tends to a finite limit as w —> 0. If we take this 
limit as the definition of the function w-?S[£; w] for» = 0, we 
obtain the spectral density of the process n(t) — <7): 


eta gig -1 pe 1 — 9(~)] [1 — 2(~)] 
Sly — <n)3 wo] = 4u-*[Kp> + <a>]? Ree BiG) Oka) , 
(6.124) 





20] = 2 SO De + <2>*Dp 
Sta — P50) = 2 OP 


We can derive a variety of special results from (6.124). For 
example, suppose the pulse length is a fixed number 79, so that 


O,(w) = ete, 6,(w) O,(w) = O(w) = <em>, (6.125) 


where 

C=pte, 
Then, from (6.124) we obtain 
Sly — ie] 





RB Re [(1 — 6)2(1 — e-t"06) (—24) eet? sin 
+27) 1—-| 8 /? 


8 
= ap Ser (6.126) 


which agrees with (6.92) for Gy = 1. Moreover, if both the pulse 
lengths and the lengths of the intervals between pulses have the 
same distribution, ie., if 


8,(w) = Ow) = Ow), (6.127) 
then formula (6.124) gives 


1 — Ow) 2. 1—[6(@) 


se] = 2 Re Le) 2 LIS) 
Sl — <i o] = Teg Re 1} fe) = aX TTF Ole) FF (6.128) 


Sometimes it is convenient to use formula (6.124) after first writing 
it in the form 
4 1 1 1 ae 
Sty - diel = Say g ey leet Teg) |) - 
(6.129) 
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* By the same method as used to derive formula (6.99), we can 
use formula (6.124) to derive the Laplace transform of. the correla- 
tion function &,(r). Writing (6.124) as a sum of two complex 
conjugate expressions, substituting into (6.95), and taking account 
of the fact that one of the resulting integrals vanishes, we find that 


ak, (7) 1 wr 1 [L — 9(w)] 1 = O(v)] 
“gr Depp o> A asso 6,(~) Ogu)” 





for 7 > 0. It follows that 


dh). 11 [1 — 6p) 1 — x6) 
UT itl-- wrap 1a ely orp 130) 





Then, using (6.98), we obtain 
) p) = —~—1__ 1 = 6.) [1 — 62()) , _<e> <o> 1 
Likes) Pl = ~ Gory pe = 6,9) 040) + +rotp'* 
(6.131) 





CHAPTER 7 


Narrow-Band Random Processes 


1. Equivalence of a Narrow-Band Process to 
a Pair of Slowly Varying Processes 


By a narrow-band process, we mean a stationary random process 
whose realizations are close to being sinusoidal oscillations of some 
fixed frequency wo, for time intervals equal to a large number of 
periods 21/w ; clearly, such a process has mean value zero. Figure 3 
shows the typical appearance of a narrow-band process, 





Fig. 3. A narrow-band process. 


In spectral language, a narrow-band process is described by the 
fact that its spectral density is negligibly small everywhere except 
in a narrow frequency band 
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where dw < wo. If 4w is the smallest such “bandwidth,” then the 


time constant 
1 


ar 


which is much larger than the oscillation period 27/w 9, characterizes 
the time it takes the amplitude and phase of the process y(t) to 
change appreciably, i.e., deviations between the narrow-band 
process y(t) and the ideal sinusoid it approximates begin to 
“accumulate” during a time of order T;. 

Suppose we are given two stationary random processes ¥,(t) and 
y(t), with zero mean values, whose rates of change are characterized 
by the time constant 7,, and suppose we form the new process 


x(t) = y,(t) cos wot — y{t) sin wot . (7.1) 


Then, the process x(t) will be narrow-band if w» > 1/7, ie., if the 
frequency wo is so large that the functions y,(t) and y,(t) do not 
manage to change appreciably during the course of a period 
27/w». To obtain the correlation function of x(#), we multiply (7.1) 
by the similar expression corresponding to time ¢ + 7, and then 
average the result, obtaining 


T, 


#%,) = (Wiz) COS eof COS welt + 7) — CH Yar) COS wot Sin wo(t + 7) 


— C¥aV ix) Sin wot COS w(t + 7) + C¥2¥er SiN wot SiN w(t + 7) 
or 


Cae) = [Cyyy 12> 608 wor — (H1Ye.> Sin wor 
+ <ya¥te) sin wor + CYaV2—) 008 wor] 
+E [Cyp¥ae> 008 w(t + 7) — Cyr Yqr) Sin wo(2t + 7) 
—KV2¥ie> SiN w(2t + 7) — CHa¥ox> COS w(2t + 7)] . 


It is clear from (7.2) that in general the process x(t) is nonstation- 
ary, since <xx,) depends not only on 7, but also on ¢. In fact, 
process x(t) is stationary if and only if 

Ni) = Weer)» 
LW = —<IIie> - 


(7.2) 
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Assuming that these relations hold, we introduce the notation 


CMY) = CH = (7), 


73 
<WYar = (7), Ae 

where s(—7) = —s(r). Then, (7.2) becomes 
<x.) = 1(r) cos wor — s(7) Sin wor . (7.4) 


Writing (7.4) in the form 
XxX) = br + is) eto + 3 (r — 15) etme", 
and taking the Fourier transform of this expression, we find that 
according to (2.9), 
Sx; 0] = p(w + on) — o(w + a4) + plw — wy) + ow — a), (7.5) 
where 
p(Q) = J 7 e'@y(7) dr = 2 fe 1(7) cos Qr dr, 
a . . . (7.6) 
o(Q) = —i J esr) dr = 2 J s(r) sin Qr dr. 
—o 0 


It is easy to see that the first of the functions (7.6) is even, while the 
second is odd: 


(—Q) = (2), o(-2) ~ —o(). (7.7) 


Since the functions p({2) and o(@) are appreciably different from 
zero only in the range 2 ~ dw = 1/7), and since wy > dw, the 
functions p(w + wo) and a(w + wo) are small for positive frequen- 
cies w > 0, Neglecting these terms in (7.5), we find that 


S[x; 0] © p(w — a) + o(w —o) (w>0). (78) 
The relations (7.7) allow us to solve (7.8) for p(Q) and o(): 


(Q) © FS[x, w» + 2) + $S[x, o, — 9}, 


(7.9) 
o(Q) & § S[x, wy + Q] — $S[x, w — Q]. 
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Thus, the functions p(@) and a() uniquely determine the spectral 
density S[x;w], and similarly the correlation functions r(7) and 
$(r) uniquely determine the correlation function <xx,>, and con- 
versely. 

Instead of two real processes y,(t) and y(t), we can consider one 
complex slowly varying process 


a(t) = y(t) + tlt), (7.10) 


which has the correlation function 


22> = Cid + HaV ee? — Kad + KI 


7A) 
= 2ir(o) = i] 
and the spectral density 
S[z;2] = 4[p(Q) + o(@)) . (7.12) 


Comparing (7.12) and (7.8), we see that the spectral densities of the 
processes x(t) and 2(t) are related by the formula 
S[x; w] ~ 4 S[z; w — w]- (7.13) 
If we multiply 2(¢) by the harmonic time factor e**o 
rapidly varying process 
2(t) = 2(t) cot, (7.14) 


, we obtain a 


which has x(#) as its real part: 
x(t) = Re &(t). (7.15) 
A comparison of (7.4) and (7.11) shows at once that 
<x.) = }Re [C22 et], (7.16) 


which is equivalent to the following formula relating the correlation 
functions of the real and complex narrow-band processes: 


xx) = $Re <28f) . 


sec. 1] NARROW-BAND RANDOM PROCESSES 181 


One can also consider the imaginary part of 2(t), i.e., the function 
X(t) = Im 2(t) = 9,(£) sin wot + yet) cos wot , (7.17) 
which is called the conjugate process of x(t) [not to be confused with 
the complex conjugate]. It is easily verified that 
<i.) = 1(r) C08 wor — 5(7) sin wer , 


. (7.18) 
<x8,.) = 7(r) sin wor + 5(r) COS wor , 


while the spectral density of the conjugate process and the cross- 
spectral density of the original process and its conjugate are given by 


S[k; w] = S[x; a], . (7.19) 
S[x, #5 «] = —ip(w + ep) + io(w + we) + ip(w — w9) + fo(w — w). 


Because of the narrow-band condition (w» S> 4w), we can write the 
last formula as 
x w= we ta(w — ay R 1; w. Or w ’ 
Sle, # a] each Pian wales ee Sal o es e 
(7.20) 
The formulas (7.1) and (7.17) establish a one-to-one corre- 
spondence between the two rapidly varying processes x(t), #(t) and 
the two slowly varying processes y,(t), y(t). In fact, solving (7.1) 


and (7.17) for y,(t) and y(t), we find that 
WE) = a1 c08 nd + 59) sino, a 
Yat) = —x(t) sin wot + H(t) cos wot . 


As is easily seen from (7.3) and (7,18), the processes x(t) and %(t) 
satisfy the same relations as the processes y,(t) and y,(#): 


(xe) = RED 
(xB) = —KeE_) = CRED. 


It can also be shown that the formulas (7.22) are necessary con- 
ditions for the slowly varying processes (7.21) to be stationary, 
provided that x(t) and #(t) are stationary. 


(7.22) 
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We have just studied the narrow-band process (7.1) constructed 
from two given slowly varying processes y,(t) and y(t). Next, we 
consider the opposite problem of finding two slowly varying 
processes y,(t) and y,(t) corresponding to a given narrow-band 
process. According to (7.21), the definition of yt) and y,(t) 
depends on the definition of the conjugate process #(t). Suppose 
%(t) is given by a stationary (i.e., time-invariant) linear trans- 
formation 


a) = | - Get —t') x’) ae’, (7.23) 


applied to the original narrow-band process x(t), In spectral form, 
we can write (7.23) as 


Ey = Fliw)x,, (7.24) 
where 
1? 
Xy = vale twtx(t) dt , 
bs eT viet opt 
Ry =e. 21) dt , 


Fin) = f ‘ e-t"G(r) dr. 


Then, obviously we have 
S[k, o] = | F(iw) ? Sl; @] , 


: (7.25) 
S[x, x; w] = F*(iw) S[x; w]. 
Comparing these formulas with (7.19) and (7.20), we see that F(iw) 
has to satisfy the relations 


—i for w—wa~ dw, 


Fw) =]; fe wins ele (7.26) 
These relations guarantee that (7.20) and (7.22) hold, which in 
turn, together with (7.21), allow us to prove (7.4), (7.18) and the 


other results given above. 
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Formula (7.26) describes the behavior of the function F(iw) only 
in the narrow intervals | w + wp» | ~ 4w where the spectral density 
S[x; w] is appreciably different from zero. Elsewhere, the choice of 
F(iw) is arbitrary. The important point is that (7.23) must trans- 
form sin wot into —cos wot and cos wot into sin wet. We now show 
two possible specific forms of the transformation (7.23) and of the 
function F(iw) which have particular advantages. 


If we choose 
F(iw) = —isgnw, (7.27) 
where 
w 
sgnw = Tol ; 


then (7.23) becomes the Hilbert transform 
ay lye? at’) 4, 1p? x(t—7 — a(t +7) 
a) =Ff oN) at ==, Carat +) dr. (7.28) 


at—t 


In this case, we can write the complex process (7.14) as 





50) = 0) +10) = pe [ta da 


= ff etse ae, 


This way of defining F(iS) has the advantage that it can be gen- 
eralized immediately to the case of processes which are not narrow- 
band. 

However, the integral operation (7.28) is not convenient unless 
we are dealing with problems in spectral form. Therefore, we now 
‘indicate another way of defining the conjugate process, which 
involves differentiation instead of an integral operation, i.e., 


: r °F (fas) x5 dew 
(7.29) 


tw 


at)=— 1s iw) = — 2 
A) =— FH), Pia) =— 2. (7.30) 


This way of defining (2) is convenient for making the generalization 
to nonstationary processes and for studying situations lying outside 
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the context ot correlation theory, i.e., situations where we use the 
apparatus of Markov processes or the theory of higher monients. 
(Only the first two moments are considered in the correlation 
theory.) The advantage of (7.30) is that it can be used in cases 
where x(t) is described by a differential equation. 

It follows from (7.30) that the complex processes (7.10) and 
(7.14) can be written as 


A(t) = xp Rex + om a(t) = (« + i) en tot , (7.31) 


Differentiating the second of these equations with respect to t, we 
obtain 
pan —imgt 53 2, 
z = e~ ot + wer), (7.32) 


an expression which is particularly convenient when the process x(t) 
is described by a differential equation of the form 


# + wix = wi f (x, %, t). 


Then (7.32) gives 
4 = —iw, e-Hot f(x, &, t), (7.33) 
where 
eieireee: 
‘ (7.34) 
ka So (zetaet — ate-tmot 


To summarize, in this section we have shown that a single 
narrow-band process x(t) is equivalent to two slowly varying 
processes y,(t) and y,(t). Moreover, we have shown how to go from 
x(t) to y,(t), ya(t), and vice versa, by using linear operations. 


2. Narrow-Band Processes Defined by Differential Equations 
In radio engineering, narrow-band processes are formed in 


frequency-selective circuits (containing capacitances, inductances 
and resistances), which are described by differential equations. 
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Moreover, the signal at the output of such a circuit will be narrow- 
band even when the external or internal fluctuations acting on the 
circuit are broad-band. Therefore, we shall consider differential 
equations containing a rapidly varying random process. We begin 
by studying the second-order linear equation 


Eek +x = f(t), (7.35) 


describing the behavior of a resonant circuit under the influence 
of the noise &(t). Here, ¢ is a small parameter which guarantees that 
the narrow-band condition dw < | is satisfied (note that w, = 1). 
In fact, it can easily be seen that the condition dw < | is equivalent 
to the condition e < 1. 

According to (7.21) and (7.30), the slowly varying random 
functions y, and y, are just 

y, = xcost —xsint, 


7.36 
Yo = —xsint —xcost. 2 


‘Differentiating y, and y, with respect to , we obtain 


dy = —(% ++) sint, 
Ja = —@ + *) cost, 
or 
Ky = &% — &)sint, 
Ig = (4 — £) cost, 
_where we have substituted from (7.35). Here, # should be expressed 
in terms of y, and y, by using the formulas 
x= y, cost —y,sint, 
—x = y, Sint + y.cost, 
which are the inverses of (7.36). As a result, instead of the single 
equation (7.35), we obtain the system of equations 
. € : € en # 
n= 5M —eésint + 591 cos 2t _ x2 sin 2t 5 
(7.37) 


4 € e . € 
2 = —ZI2— cost — 39, sin 2t — 592608 2t. 
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The appearance of the small parameter « on the right in (7.37) 
confirms that the processes y, and y, change slowly. We shall say 
that the equations (7.37) are in standard form, and we shall refer to 
the double-frequency terms 


5y.cos2t, 5yesin 2t, 59. sin 2t, 52008 2t 


in (7.37) as the oscillatory terms. The effect of these terms during a 
“period” of the processes y, and y, is small and can be neglected in 
the first approximation; thén, the equations (7.37) reduce to 


a € . 
n= —3n—eésint, 
(7.38) 
p € 
Je = — 792 — eE cost. 


If ¢ is sufficiently small, we can investigate the equations (7.38) by 
using the method given in Chap. 4, Secs. 7-9; this corresponds to 
replacing the actual processes y,(t) and y,(t) by Markov processes. 
Using this method, we can replace (7.38) by the equivalent 
“simplified’’ equations 
rA= — 5% — by 
; (7.39) 
de> ~ 5N2 — «2, 


where €, and , are delta-correlated random processes with zero 
mean values and correlation functions 


Céxkis> = 80) [” <€E,) sin sin ¢ + 2) dr, 
KExbn) = 8(r) [”_<Efs> cost cos (t+ 7)dr, (7.40) 


Clabes = atr) f - CEE.) sin t cos (t +1) dr. 
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Each of the integrals appearing in (7.40) can be written as a sum 
of two integrals, e.g., 


ix <éé, sin t sin (t ++ 7) dr 
- 3S <> cost dr — an ££, cos (2¢ + 1) dr. 


Here, the second integral in the right-hand side can be neglected in 
the first approximation, since it represents a nonstationary 
oscillatory contribution, which has only a slight effect on the 
Fokker-Planck equation when averaged over a period. The same is 
true of the other two integrals in (7.40), from which the oscillatory 
parts should also be subtracted. As a result, instead of (7.40), we 
have 


<£ibte> = (bebe? = Ke8(7), 
<hfe> =0, 


(7.41) 


where 
1/2 1 
K,=5 J _ So cos r dr = 4 STE I]. (7.42) 
It follows from (7.39) and (7.41) that y,(t) and y,(t) are identical 
exponentially correlated random processes. In Chap. 4, Sec. 10, it 
was shown that the correlation function and spectral density of such 
a process are given by (4.217) and (4.212), respectively. Applying 
these formulas to (7.39), i.e., setting 
B= 33 K=€K), 
we find that 
Wid = Ped = (7) = Keel? 
“Wer = s(r) = 0, (7.43) 
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Moreover, according to (7.4) and (7.8), we have 


<xx,> = eKge-"l/2 cos wt , 
éK, 
(= wo) + (A) 


Of course, these formulas could have been obtained directly from 
equation (7.35), The approach used here is convenient since it can 
be generalized to the case where the original equation is nonlinear. 
Such nonlinear equations will be studied in Vol. II, Part 2, which is 
devoted to the statistical theory of nonlinear oscillations, and 
methods will be given there for constructing higher-order approxi- 
mations in the parameter e. In the generalization to the nonlinear 
case, the slowly varying processes y,(t) and y(t) remain Markovian, 
a fact which is very important, since this ensures that effective 
methods for analyzing the processes are available. 

* Everything just said can also be generalized to the case of 
equations of higher order, both linear and nonlinear. To illustrate 
how to go about making this generalization, we consider the 
fourth-order differential equation 


(t+ ep tl tA) teptl— dx =e), (7.45) 


where « and 4 are constants, and p = d/dt. This equation describes 
the behavior of two “weakly detuned” resonant circuits in series, 
and has the expanded form 


(7.44) 


S[x; e] = (w > 0). 


(So +1) @ +5) $28 + 8) + AG — Be) = EQ). 
,Generalizing (7.36), we introduce four slowly varying processes: 


yy = xcost —xsint, 


Ye = —xsint — zcost, 

y= = F* cot 2+ sine, (7.46) 
o sh 

w= Z+* sint —2+* cost 
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The inverse of the system (7.47) is 


x= y, cost —y,sint, 

—* = y,sint + y,cost, 

Ex 
€ 





= yy cost — y,sint, (7.47) 





= = = ygsint + y,cost, 


Differentiating (7.46), we obtain the relations 


Wy = —(¥4+%)sine, 

Jo = —(% +x) cost, 

“ ¥+x7.. 

de = -[((4+1) : ] sine, (7.48) 


i= [fet Eee, 








It follows from (7.45) and (7.48) that 


Js 


n= 


xe 





= -{-274 +(1 +4) 2—#—% + é]sint, 
(7.49) 
=-{-2 SHA 4 aye —#— 24 e) cost, 


Of course, we must next express the right-hand sides of (7.49) in 
terms of y,, ¥2 3 and y,, by substituting from (7.47). As a result, 
we obtain the following exact equations in “standard form”: 

{ 


A= 
a= 
js = 


n= 


—e(ys cos ¢ — y, sin t)sint, 

—e(ys cos t — y, sin t) cost, 

—e[(2(y sin t + y, cos t) + (1 + 4?) (y, cos t — y, sin t) 
—e(ys cost — y, sin t) + é]sint, (7.50) 
—¢[2(yg sin t + y, cos t) + (1 + 4?) (y, cos t — y, sin £) 
—e(y, cost — y, sin t) + é] cost. 
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Then, in the right-hand sides of (7.50), we have to go over to 
sums of sines and cosines at the zero and double frequencies. In the 
first approximation, we can neglect the double-frequency “‘oscil- 
latory terms,” as well as nonoscillatory terms of higher order, 
which are proportional to e*. Thus, in the first approximation, we 
finally obtain the following system of equations: 


y =f 

yn = 5M» 

. € 
J2= — 7s» 


(7.51) 
= 91 +5 (1 +4) y.— ef sine, 


S 
\ 


n= —%—5(1 + 4?) y, — ef cost. 


The equations (7.51) are of the form (4.185), and for small e, the 
Processes 1, Yo, Ys, Ya Can be regarded as Markov processes. This 
is equivalent to replacing the noise terms sin t and £ cos ¢ in 
(7.51) by delta-correlated processes ,(t) and £,(t) with the prop- 
erties (7.40) which go over into (7.41) after discarding oscillatory 
terms. The resulting system of equations separates into two 








completely independent systems, In fact, if we write ys = —yq, 
these two systems 
ya Ya 
€ 2" 
* (7.52) 
2 =e 5 di — Sas 
and 
da Vs 
€ 2° 
he (7.53) 
2 =—¥s— ‘ ye § 


are entirely identical, 
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From (7.52) and (7.53), it is easily deduced that each of the 
processes y,(t) and y,(t) has the spectral density 


= Ko 
ow) = aay = + SADE awe” 





ow) =0. 


Then, according to entry 5 of Table 1, p. 25, we have 


K, 
re) = 37 7a 





fit 4 1. «4 

ev! WA (cos Sr + 5 sin $71). (7.54) 
In the case where the equation (7.45) is nonlinear, the equations 
for the slowly varying processes y,, 2) 3, 4 Will also be nonlinear. 
However, the apparatus of Markov process theory can still be 
applied to these processes. * 


3. The Amplitude and Phase of a Narrow-Band Process. 
Rayleigh Processes 


The slowly varying processes y,(t) and y(t) considered above are 
convenient because they are linearly related to the original narrow- 
band process x(t). However, instead of y,(t) and y,(t), it is some- 
times appropriate to consider the amplitude A(t) and the phase 9(t) 
of the process x(t), These slowly varying processes have an obvious 
intuitive meaning, and are defined by the relations 


A=V#RTH, =are tan 3 (7.55) 
1 
with inverses 
: Wn=Awsp, ¥%=Asing. (7.56) 


The processes (7.1), (7.17), (7.10) and (7.14) considered previously 
can be expressed in terms of the amplitude and phase as follows: 


x=Acos(wtt+op), #= Asin(wet + 9), 


z= Ae, z= Aemitto, 


(7.57) 
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The amplitude and phase are related to x(t) in a nonlinear fashion. 
Therefore, to calculate their statistical characteristics, we have to 
know the probability distribution of the process x(t). For simplicity, 
we shall only consider the case where x(t) is assumed to be a 
Gaussian process. Then, knowledge of the correlation function 
(7.4) is sufficient to determine all possible distributions not only 
of x(t) itself, but also of the slowly varying processes y,(t) and 
y(t). In fact, y(t) and y,(t) are also Gaussian with zero 
means, since they can be expressed linearly in terms of x(t). 
Once we know the correlation functions <y,y1.> = <Yave = 7(7) 
and the cross-correlation function <¥,y2,> = s(7), we can find the 
various probability distributions of y,(t) and y,(t). 

Since s(7) is odd, the cross correlation between the values of 
y(t) and y,(t), taken at the same time t, equals zero, and hence 


AY» Ya) = aa ec tutt yb Ro? , (7.58) 


where o? = r(0). Making the change of variables (7.56) and 
integrating over all possible values of the phase from 0 to 27, we 
find that the amplitude has the Rayleigh distribution 


w(A) = Ed e~A ot (7.59) 


A random process A(t) which is the amplitude of a narrow-band 
Gaussian process is called a Rayleigh process. 

To find the two-dimensional probability density w(A, A,) of the 
amplitude, we have to use the distribution of the random variables 
Ye Jr» Yar, Which can be found from their correlation matrix 


7(0) 0 r(r)  s(r) 
i) 7(0) —s(r) 17) 

r(r) —s(r) 70) O 

2) (r) 0 (0) 
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Then we go from yy) Yes Vie Yor to A, Az, Y, Ye, and integrate over 
all values of p and ¢,. As is well known,! the result is 


WA, A.) = Abr 1, (2 AA) exp |— =A AE |, 7.60) 








o(1 — Q%) 1—Q? o, 26%T — 0%) 
where 
9- OO, a=), 


and I,(z) = J,(#z) is the modified Bessel function of the first kind, 
of order zero. 

:In the case of a narrow-band noise process described by a 
differential equation, the behavior of the amplitude and phase is 
also described by differential equations, just as in the case of the 
functions +, and y,. For example, consider the process described 
by the equation (7.35). Differentiating (7.55), we find that 





i Aan + Wt ge we Vey (7.61) 


Substituting (7.39) into (7.61) and using (7.56), to the first approxi- 
mation we obtain the equations 


A= —54 — «£, cosy — ef, sing =«F,, 
(7.62) 


¢ = 4 lhsine — £,cosg] = Fe, 


where £,(t) and £,(t) are Gaussian processes with the properties 
(7.41). 

* To write down the Fokker-Planck equation for the amplitude and 
phase, which vary in the way described by (7.62), we apply formula 
(4.194). First, we calculate the coefficients appearing in (4.194). 


7 1See e.g., W. B. Davenport, Jr. and W. L. Root, An Introduction to 
the Theory of Random Signals and Noise, McGraw-Hill Book Co., Inc., 
N.Y. (1958), p. 162. 
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It is easily seen that 
0 0 
J _KUFL Fiddr= [<6 008 @ + fain 9) (E008 » + far sin g)> dr 


0 0 
= costp f <Eifisd dr + sinty | (Eabas> dr 





; (7.63) 
=}K,, 
and similarly, 
I * KlF» Fale = sar Ko, 
f. ° KIF,, Fr] dr = 0, 
(7.64) 


[KIB a] = af asine — foo) 
x (€1, Sin p — &y, Cos p)> dr 


1 

=5q%- 

Thus, according to (4.194), the required Fokker-Planck equation 
has the form 


‘ Ae é aw 1 Ow 
mAs) = -<aal(-F + 3g Xe) #] + 5 Kol age + ae Apel’ 

(7.65) 
which corresponds to the following fluctuation equations for the 
amplitude and phase: 


2 
A=-SAt 5G Ko teh, 
(7.6) 


pad 


Here, £, and €, are the same kind of independent random processes 
as in equations (7.39) and (7.62), with the same properties (7.41). 
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: Although the term <*K,/2A is of order 2, we have retained it in 
(7.66), because the intensity coefficient K, of the fluctuations may 
bé quite large. It is best to assume that K, is of order | /e, and write 


K,=*. 


Then all the terms in (7.65) are of the first order in ¢, and it is 
convenient to write the noise terms in (7.66) as /€n, and +/ens. 
As a result, the first of the equations (7.66), which describes the 
behavior of the amplitude, becomes 


€ 
A =-5(4-4)+ven, (7.67) 
where 
. p> =90, <ai> = «8(r)- 
The one-dimensional Fokker-Planck equation corresponding to 
(7.67) is 


ban = Ala A+R 0.9 


We already know the form of the stationary solution (4.49) of the 
arbitrary one-dimensional Fokker-Planck equation (4.33). Applying 
(4.49) to (7.68), we find that the stationary amplitude distribution is 


w(A) = Ceta2atind — Aan, (7.69) 
which, of course, is the sante as (7.59), with x = 0. 

Since the changes in amplitude represent a Markov process, the 
expression for the transition probability p(A,, A) is of great 
interest, since all possible probability distributions, in particular, 
the two-dimensional distribution 


; (A, A,) = p(A,, A)a(A), 
can be expressed in terms of P(A, A). Using (7.60) and (7.59), we 
obtain the following expression for the transition probability 

















_ A, Ay) 
ae a 7) 

_ A, P Aa.) i ame (7.70) 
: = a OF MT — G8 ot OP I 2051 — OF) 
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Because of the relations (7.43), which were found earlier for the case 
described by the equation (7.35), the coefficient Q(z) in (7.70) has 
the form : 


O(n) = ere, (7.71) 


If Q(7) has a different form, A(t) will not be a Markov process, and 
the conditional probability (7.70) will not play such an important 
role as in the case of a Markoy process. Substituting (7.71) into 
(7.70), and replacing o? by «, we find that the nonstationary 
amplitude distribution is 


A AA a AR 
4) =e! (aeinh capa exp| Sql eo) Ter 


(7.72) 


whose time variation is described by (7.68) if the initial distribution 
is a delta function, i.e., if 


w(A) = 6(A — Ay) for + =0. 


This is one of several examples where a Fokker-Planck equation 
with a nonlinear ‘average velocity” K, (cf. p. 58) has an exact 
nonstationary solution, in the form of an analytic expression 
involving elementary functions. 

It is sometimes required to find the statistical characteristics 
of a narrow-band random process x(t) or of the other processes 
(7.56) and (7.57), given the statistical characteristics of the processes 
describing the variation of amplitude and phase. In its general form, 
this is a complicated problem, since the transformation from A(t) 
and g(t) to the processes mentioned is nonlinear. However, the 
problem can be considerably simplified in the case where the 
amplitude and phase processes are statistically independent of 
each other. We now show how to calculate the correlation functions 
of the processes 


2(t) = Aer, xt) = A cos (wot + ¢) 
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in this case. Using the independence of the amplitude and phase, we 
can write the correlation function 


' 


: (228) = (AA, e990) (7.73) 
as 

<2a2t) = (AAD <elto-eey . 
The average <e“'?-*")» is the value at u = — | of the characteristic 
function 


8,(u) = <emtor-my (7.74) 
‘of the phase increment g, — . Therefore, we have 
<z2t) = <AA, 8,(— 1). (7.75) 


‘Then, using (7.16) we find that the correlation function of the real 
jrandom process x(t) is 


, <xx,) = $<¢AA, Re [O,(— 1) et]. (7.76) 


When the phase increment has a symmetric distribution about 
yzero, the characteristic function @(w) is real and (7.76) becomes 


<x, = $¢AA,) O,(— 1) c08 wor. (7.77) 
‘According to (7.4), this means that 
1(t) = §<AAD O(—1), (7) = 0. (7.78) 


As an example, consider the case where the phase changes represent 
a Wiener process (p. 135), i.e., a “pure diffusion process.” Then 
9, — pisa normally distributed random variable, whose mean value 
is zero and whose variance grows linearly with time: 


Diy, — 9) = KgI71, 
‘In this case, 
6,(u) = exp {— } K, |r| u?} 


and formula (7.77) becomes 


.<xx,> = $<AA, e-Kolt!/2 cos wor . (7.79) 
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4. Quasi-Rayleigh Processes 


Suppose we are given a narrow-band Gaussian process (ft) with 
correlation function o?R(r), and suppose we add to &(t) a “‘sine 
wave” E cos (wot + 3), with fixed amplitude £ and random initial 
phase % which is statistically independent of &t). Then, the 
resulting process 


x(t) = E(t) + Ec0s (wot + 50) (7.80) 


is no longer Gaussian. For (7.80) to be a stationary process, it is 
necessary that the initial phase 9 be “completely random,” i.e., 
that it have the uniform distribution law 


1 
w(8e) = 5 (7.81) 
Then, using the statistical independence of &(t) and E cos (wot + 9), 
we can easily find the correlation function and characteristic 
functions of x(t): 


Gen) = o8(r) + 5 cos wer, 


<eitty = ert Ju), 
; (7.82) 
<eimevinetey = exp |S (ut + 2uu,R(*) + u2)| 


x JoE u? + u? + Quu, cos «,7) ‘ 


where J,(2) is the Bessel function of the first kind, of order zero. 
The narrow-band process x(t) can also be written as an oscillation 


of the form 
2(t) = B(t) cos [wot + ¥(t)] (7.83) 


with slowly varying amplitude B and phase ¥, which we denote by 
new letters to distinguish them from the amplitude A and phase 9 of 
the Gaussian process discussed in the previous section. We now find 
the probability density w(B). First, we write the process &(t) as 


& = J, C08 wot — ¥z SiN wot , (7.84) 
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where y, and y, are random variables with the normal distribution 
law (7.58). Substituting (7-84) into (7.80), and comparing the 
result with (7.83), we have 
y, + Ecos, = Beosp, y,e+Esind, = Bsing, (7.85) 
so that 
Vi + yi = B? + EF? — 2BE cos (¥ — 9). (7.86) 
By making the change of variables (7.85), we go from the distri- 


bution 


I 2 4 y2 
Wr Ye, Fo) (re)? exp | roa (7.87) 





to w(B, , 3). Bearing in mind that 
dy,dy,d3, = BdBapdd, , 
and substituting (7.86) into (7.87), we find that 


w(B, Y, 94) = exp |— aig [Bt + B* — 2BE cos (W — 90). 


B 
(Qro}® 
Then, integrating this expression with respect to y and 4, we 


‘obtain the amplitude distribution 


0) = Ze |- PAP Hs (22), 


= (7.88) 


which is a generalization of (7.59). A calculation of the mean value 
co 
(BY = J w(B) BaB 
0 
leads to the following expression: 
ERgn B B EX.) Ee 
= fortis Ze) n(x) + eh Eel). 089 
The random process B(t) will be called a quast-Rayleigh process. Its 


importance in radio engineering is due to the fact that a received 
radio signal often contains both noise and a useful signal which has 
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the form of a sine wave whose amplitude can be regarded as 
constant, for simplicity. 

There is a different way in which one can arrive at a quasi- 
Rayleigh process. Suppose that in formula (7.1), y(t) and y,(t) 
are stationary Gaussian processes whose mean values are not zero. 
Then the process x(t) will be nonstationary. However, stationarity 
can be achieved by making the time origin (i-e., the time to which 
all other times are referred) indeterminate; in fact, it is sufficient 
to introduce a physically imperceptible error in the location of the 
time origin, with order of magnitude equal to the high-frequency 
period 27/w). Then the amplitude of the process (7.1) will be a 
quasi- Rayleigh process with 


E= Vy + (y2)?. 


The quasi-Rayleigh process B(t) is not a Markov process, even 
when y,(t) and +,(t) are exponentially correlated random processes. 
Instead, B(t) is a component of the two-dimensional Markov 
process (B(t), ¥(t)) decribed by the equations 


B B oa E 

eT OZ tag ty ex th, 

; : 7.90) 
BE. .¢ ( 

X= —sanxt 3 (x =4—4), 


which generalizes (7.66). Here, ¢, and £, are independent Gaussian 
delta-correlated processes such that 


ED = (ED =0, ibd = Eafed = 28). 


However, for simpliticy, in solving certain problems it turns out to 
be convenient to replace the quasi-Rayleigh process B(t) by the 
one-dimensional Markov process corresponding to the equation 

B Bo 
af oe © 


€ 


E 1(EBjo?) 
TZ T(EB/o) 





+&, (7.91) 


which has the same stationary distribution as (7.88). This replace- 
ment is satisfactory both for large and small values of E/c. 


PART 2 


Nonlinear Transformations 
of Signals and Noise 


CHAPTER 8 


Zero-Memory 
Nonlinear Transformations 


In studying nonlinear transformations of random signals, just as 
in the case of nonlinear oscillations in general, it is difficult to 
develop a universal practical method of solving problems. Instead, 
different problems require the use of different methods, which are 
justified in each case on physical grounds. It is appropriate to 
distinguish two basic cases, depending on the character of the 
nonlinear system in question: 


1. The simplest nonlinear systems are those for which the value 
of the output function y(t) at any instant of time is determined 
only by the value of the input function é(¢) at the same instant of 


time, ice., 
nt) = s{E1, (8.1) 


where g(£) is a nonlinear function (see Figure 4a), Such a non- 
linear transformation is said to have zero memory (o7 to have no 
inertia). A closely related group of nonlinear transformations are 
those such that the input function ¢(¢) is subjected to an additional 
transformation by a linear system Sj, and the output function is 
subjected to an additional transformation by a linear system S3, 
where the action of the systems S, and S, is not affected by the 
presence of the nonlinear device (see Figure 4b). Such a trans- 
formation can be written in the form 


a(t) = LeglLié()] , (8.2) 


where L£, and L, are the linear operators describing the behavior of 
203 
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the systems S, and S,. We know how the correlation functions 
and quasi-moment functions of random signals transform when the 
signals undergo linear transformations. Therefore, the study of 
transformations of the type (8.1) is equivalent to the study of 
transformations of the type (8.2). 





(b) 


Fig. 4. Examples of nonlinear zero-memory transformations. 


2. A more general and complicated class of transformations 
consists of nonlinear transformations with memory (or inertia), in 
particular, transformations described by nonlinear differential 
equations. In Figure 5, we show one of the simplest examples of 
such a transformation, i.e., a system whose behavior is described 
by a first-order nonlinear differential equation. Let £(t) and 7(t) be 
the random input and output voltages, respectively, and let D be a 
nonlinear element (a diode) whose current-voltage characteristic is 


1=FV), 





Fig. 5. The diode detector, an example of a nonlinear transformation 
with memory. 
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where V = £— 7. Assuming that the internal resistance of the 
input-signal generator is zero, and using the obvious relations 


1 
h+h=1, =qfha=nr, 


we obtain the differential equation 
d 1 
Oy d= are (8-3) 


It is difficult to solve this nonlinear equation in the general case 
where the function F is arbitrary, even for a deterministic input £, 
since the solution cannot be expressed in terms of a finite number of 
integrals. Therefore, it is natural that in the case of random 
functions, the solution of (8.3) involves great difficulties. 

We now make a few remarks about the form of the function g(£) 
appearing in (8.1) and (8.2). This function is called the transfer (or 
dynamic) characteristic of the nonlinear device, and is usually 
determined experimentally from the current-voltage characteristics 
of nonlinear elements like vacuum tubes. Then, to facilitate 
subsequent calculations, one makes some suitable approximation to 
g(£). Three approximation methods which are convenient for 
theoretical purposes are to approximate g(f) by 1) a polynomial, 
2) a broken line (the so-called piecewise linear approximation), and 
3) by an exponential. Each of these methods has its advantages and 
disadvantages. It should be kept in mind that the requirement that 
the approximation be accurate and the requirement that the 
resulting analytic expressions be simple are in a certain sense 
antithetical, and as a rule incompatible. 

In the next two chapters, we shall analyze what happens to 
random signals when they are acted upon by nonlinear systems. 
Our problem can be formulated as follows: Given the parameters of 
the system and the statistical characteristics of the input signal €(t), 
find the statistical characteristics of the output signal y(t). 


t Cf. P. I. Kuznetsov, R. L. Stratonovich and V. I. Tikhonov, Passage of 
some random functions through linear systems, Avtomat. i Telemekh., 14, 144 
(1953); Passage of random functions through nonlinear systems, ibid., 14, 
375 (1953). 
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1. Transformations of Probability Densities 


For nonlinear zero-memory transformations of the form (8.1), 
the basic solution of the problem just formulated is a consequence 
of the following familiar result: Suppose we know the n-dimensional 
probability density w,(é,, ..., &,) of the random variables 


£ = EAs rr En = Ete) 


and suppose we want to find the probability density w,(7,, ---» 72) 
of the new random variables 


m = Bibs fn)» 
: (8.4) 


mceplbe EA 


where the functions g,,...,g, are piecewise continuous. If the 
inverse of the system (8.4) is 


& =A on tH), 
(8.5) 


En = Inlay +> In) > 


where the functions /,,...,4, are single-valued, then the pro- 
bability density w is given by the formula 


Wir oo+5 Me) = WelAy(, -o-2 Mads os Aas ve Mad | Dal, (8-6) 


where D,, is the Jacobian of the transformation from the random 
variables €,, ..., &, to the random variables 7, -... In» Le., 


thy ah 
a" On, 
Di=| + wm . |. (8.7) 
ay hy 
dn, Onn 
In cases where the inverse functions /y, ..., 4, are not single-valued, 
we have to sum the right hand side of (8.6) over each of 
the subregions involved. For the one-dimensional case, this 
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transformation rule was derived on pp. 9-10. Applying (8.6) to the 
case of transformations of the form (8.6), we obtain 


%, (ms 12) = welh(n), A(na)] | AO) A'Cne) | (8.8) 


forn = 2, where = h(n) is the inverse of the function g(£). 

Difficulties can arise in applying formulas (8.6) and (8.8) to the 
nonlinear zero-memory transformations of practical interest. For 
example, if the transfer characteristic g(£) in (8.1) is a polynomial 
of degree higher than three, then it is hard to find the inverse 
function A(n) in general form, i.e., it is hard to solve (8.1) for & 
analytically. Thus, in this case, it is appropriate to approximate the 
tunction g(é). If we make a piecewise linear approximation to g(£), 
in many cases the function 4(y) turns out to be discontinuous 
with an infinite derivative h’(y) at certain points. Then, at these 
points, the probability density of x(t) will haye delta-function 
discontinuities. 

We now consider a few special cases. Let 


m = (4) = &> ne = &2(£ £2)» (8.9) 
where the inverse functions 
£, =A(n) = Ww £2 = han, 12) (8.10) 


are single-valued. The Jacobian of this transformation is 


. 


1 0 a, 
P=! ag ~ahn |= One’ 
am One 
and therefore 
ah 
20,( me) = wef har 7] | | ; (8.11) 


To obtain the one-dimensional probability density of the random 
variable 72, we integrate (8.11) with respect to 7): 


tn) = few (ns 1) | Go| (8.12) 
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Using (8.12), we can derive the following expressions for the 
probability densities of the sum, difference, product and quotient of 
the two random variables £, and £,: 





wo, + ba) = wo(n) = f meen 1 — £1) der (8.13) 
wa(, ~ £2) = mn) = [web & — 1) dbs (8.14) 
tlErée) = ohn) = fe(En 2) (, (8.15) 
wo) = wn) = f lla, fs) | db (8.16) 


If the random variables €, and €, are independent, with probability 
densities p(£,) and g(é,), then in these expressions we have to set 


welE1, £2) = P(E) (Es) - (8.17) 


The next two examples illustrate the use of formulas (8.15) and 
(8.16). 

1. Suppose we want to find the probability density of the 
product of two correlated random variables £, and £,, whose joint 
probability density is normal, i.e., 





1 1 2 2 (REE. 
wb bd = 5a | mary (es — 2Riiés)| 


oa 


Then, applying (8.15) with » = &&,, we obtain 


= 1 {Ry 
mn) = VT? tool a | 





es [exp | Ta = pe (f+ a ){ a 6.18) 


1 R 
~ F040, V1 =e | axl sam) | Ke ( sare Re )i 
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where K,(2) = (7i/2)H{(iz) is the modified Bessel function of the 
second kind, of order zero.? 

2. Next, we calculate the probability density of the ratio 
n = &/&, of two normally distributed, correlated random variables 
é, and ,. According to (8.16), we have 





= 1 - & 7 1 i 2Rn 
“0) = ae |, exp} ~ a) (3 Pie ){ be, 
ae VIR 1) 


= a(o2 — 2Ro,0, + of?) 7 


2. Moment Functions for Polynomial Transfer Characteristics 


Let the transfer characteristic 7 = g() of the nonlinear device 
be an analytic function in a certain interval, so that it can be 
expanded in a Taylor’s series 


af) = a + 4,(€ — 0) + 4(f — 0)? +..., (8.20) 
where 


a= gene . 


Then, depending on the required degree of approximation, we 
retain a certain number of terms in (8.22), say the first 2 + 1 terms: 


&(£) =Pag + a,(& —c) +... + @(E — 0)”. (8.21) 


Even if this approach is not possible, we can still always find a 
polynomial which approximates the function g(£) with the required 
accuracy for a given input process £(t). In choosing this approxi- 
mation, it is important that the polynomial be a good approximation 
to g(é) on that part of the ¢-axis corresponding to values of &(t) 
which have high probability. However, the approximation can be 
quite inaccurate for parts of the transfer characteristic corre- 
sponding to values of (t) which occur with relatively small prob- 
ability. 


® See I, M. Ryshik and I. S. Gradstein, op. cit., formula (6.447.2), p. 318. 
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Another way of finding the best approximation to g(£) is based on 
the method of least square error and goes as follows: If the 
approximating polynomial is to have z + 1 terms, we first form the 
difference 


#0) — Dalg — oF. 
k=0 


Then the unknown coefficients a, are determined from the 
condition that the integral of the square of the error, Le., of the 
quantity 


r={ : [ee +) — dae]: de, (8.22) 


should take its minimum value. The limits of integration a and b are 
chosen in such a way that the yalue of the random variable € is 
overwhelmingly likely to fall in the interval a+¢ <é<b+c, 


which means that 
+c 


wé)déxl. 

ate 

If the input process ¢(t) is specified by its moments, then without 
using these moments to reconstruct the probability density w(), 
it is convenient to set 


c=m, b= —a =A Vm, — mi. (8.23) 


Moreover, if the probability density «(£) does not differ too strongly 
from a normal distribution, we can choose A equal to about 2 or 3. 
The necessary conditions 


al al 


Ba, ren 5g = 0 


for the expression (8.22) to have a minimum lead to the system of 


equations 
n. — (—]yretr b 
ie bitita, = [ xig(e + ode, (8.24) 
2 : 


from which the coefficients 4p, a, ..., d, can be determined. 
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Thus, for the rest of this section, we shall assume that instead of 
the general transformation (8.1), we are dealing with a trans- 
formation of the form 


y(t) = ayx(t) +. + a,x"(t) , (8.25) 
where 


HA = at) — 40, H(t) = E(t) —€ = &t) — &). 


To find the moment functions of y(t), which will be denoted by 
f,, we multiply the expression (8.25) by similar expressions 
corresponding to different instants of time. We then average the 
result, obtaining 


Fiy(t) = agperlt, £) + aateg(t t,t) + oe + dattn(ly ost), 


Fig (ty, te) = Qiptg(ty, ta) + MeL Halts, fay fa) + paltry a» 2)] 
(8.26) 


yal prg(tys tay bay ta) + alta tay f1, ta] 


+ agnig(ty, ty fey ts) +, 


Here, the quantities 
Paltay sey fn) = (X(t) --+ (tn) 


are the moment functions of the process x(t), and are equal to the 
central moments of é(t) whence = <£(t)>. 

It is apparent from the formulas (8.26) that the moment functions 
of the output process (ft) can be expressed as linear combinations 
of the moment functions of the input process ¢(t). However, the 
formulas for the moment functions of the output process involve 
higher-order moment functions of the input process. This is 
one of the characteristic features of nonlinear transformations as 
compared with linear transformations. We now consider some 
examples. 
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Example 1. Let the nonlinear device have the transfer 
characteristic 


Ht) = g(€) = ag%(t) — a,£X2), (8.27) 


and let the input £(t) be a stationary Gaussian process with zero 
mean and correlation function 


R(r) = o?R(r). 


We wish to calculate the mean value and correlation function of 
y(t). It is easy to see that the first and second-order moment 
functions of y(t) are 


WO) = @F()> — aa HO) » 
<Wt) (te)> = 4K Py) Plt)? + aiCE(4) E> (8.28) 
~ agal$(t1) EX(t2)> + <é(h) P(4)>] - 


As is well known, the univariate moments of a Gaussian process 
like é(t), with mean zero and yariance o*, are given by the formula 


_{1-3+5++-G7— ot for evenj, 
<> = Jo for odd j. (8.29) 
To calculate the bivariate moments of the normal process &(t), we 
use the expansion (3.14), from which it follows that 


a. yt 
Leia) = oitk > NyNnr Ro) ’ (8.30) 
imo 





where 


Nn = r BFOHN(x) de, 


The set of coefficients Vy, forms a matrix of the form 
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1 2 3 4 5 

0 0 0 0 0 

-1 0 0 0 0 

0 21 0 0 0 
31 0 |-321/ 0 0 

0 | 43-1 0/4321] 0 
—531! 0 |-5431) 0 |—5-4-3 

2 











9 6-5-3+] 9 6-5-4-3-] 0 


























This matrix has the following properties, which are easily verified: 

a) All elements above the principal diagonal (j < 2) vanish; 

b) The only nonzero elements on or below the principal diagonal 
(j > k) are those whose indices j and k have the same parity; 

c) The nonzero elements Nj, are obtained by first forming the 


product ; : 
IG -N)NG-2---G-k+N 


containing & factors, and then multiplying the result by all odd 
numbers from j — k — | to l, eg., 


Ne =6°5-3-1; 
d) The sign of N,, is negative if k is odd and positive if k is even. 
Applying these rules and (8.30), we find that 
(PED = of{l + 2R%(7)] , 
PED = [3 + 12R%7)], (8.31) 
KEE = 0[9 + 72R%(r) + 24R(7)] . 
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If we use (8.29) and (8.31), then, according to (8.28), 


<¥> = ago* -— 3a,0*, 
Op reat Pea (8.32) 


<yy,> = aot + 2R?) — 6a,4,0%(] 4-4R?) + a20%(9 + 72R® + 24R*). 
In particular, the variance of y is 


Dy = <3) — (9)? = 2a2ot — 24a,a,0% + Major . (8.33) 


Example 2. Let the nonlinear transformation have the para- 
bolic transfer characteristic 


Ht) = ayx(t) + aax7(t) , (8.34) 
and let the input signal x(t) be the sum 
x(t) = s(t) + &(t) (8.35) 
of the “noise” é(¢) and the “useful signal” 
s(t) = Ecos (wt + 90) s (8.36) 


which is a sine wave of fixed amplitude and uniformly distributed 
initial phase. As in the preceding example, we assume that £(t) is a 
stationary Gaussian process, with zero mean and correlation 
function k(r) = o?R(z). 

Substituting (8.35) into (8.34), multiplying the result evaluated at 
time ¢ by the similar expression evaluated at time ¢ + 7, and then 
taking the average, we obtain 


(> = ax) + aX), 
CW) = ACS.) + AKEED + aR + 40s, LED gy 
+ Dahist> <E> + akCePet> . 


In deriving the relations (8.37), we have used the fact that the 
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statistical independence of the random functions s(t) and &(t) 
implies the relations 


(sE> = (SED, (57%) = Cs) CED, 

«ss,£6.> = (ss, CEED, (562) = Cs) (BD), ... 

Then, since 
O=G=0, ©=é>=0, 

we have 

sg >=0, E> =0, (8s, £) =0,... 
We have also used the relations 

is) =0, <(&&>=0, 

from which it follows that 


Caxt) = Cy) = 0. 


Since we already know that 
<€E> = oPR(r), (£72) = of] + 2R%(r)] 
[cf. the first of the formulas (8.31)], then, to evaluate (8.37) we need 


only calculate the moments of the useful signal (8.36). Transforming 
the expression 


(55, = E*Ccos (wot + 9) COS (wot + wt + Go)» 
into the form 
ss.) = $ E*{cos (2ugt + wor + 2p) + COS wor) , 


we observe that the first term in the right-hand side vanishes when 
averaged with respect to the initial phase yy, whereas the second 
term is not random and remains unchanged. Therefore, we have 


<55,) = 4 E® cos wor , (8.38) 
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and similarly 


Gs'st> = FELL + cos 2(wgt + g9)] [1 + cos Awol + wor + vo) 


= 4 £1 + 4008 2ugr) . (8.39) 


Substituting (8.38) and (8.39) into (8.37), we obtain 
<y) = $4,E* + ao?, 
<9.) = 4.4}E* cos wor + afo*R(r) + 4aZE“(1 + $008 2wyr) (8.40) 
+ 2ado®E*R(r) cos w,r + ao%E? + ado*{] + 2R%z)] , 
from which it follows at once that the correlation function of the 
output signal is 


Ar) = 4a cos wr + ajo®R(r) + $ a3 E* cos 2w, 7 


(8.41) 
+ 24203 E*R(r) CoS wr + 2a3ctR%(r). 


Example 3. Let the transfer characteristic of the nonlinear 
device be 
y =a(A—c) +4,(A — ec) (8.42) 
or 
y-a=aA+a,A’, (8.43) 


where 

aq=ae—ac, a,=4a,—2ae. 
Here, we assume that the input A(t) is a Rayleigh process, which is 
the envelope of some narrow-band Gaussian process (see Chap. 7, 
Sec. 3), and has the distribution described by (7.59) and (7.60). 
From (8.43), we find in the usual way that 


Y= a+ aK Ay +064, (8.44) 


<(y — 4%) 1, — 45)> = a AA,> + aja,{( AA? + ¢A2A] + a3 AP AD. 
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To calculate the moments of the random function A(t) which 
appears in (8.44), we use the distributions (7.59) and (7.60). First, 
we introduce new variables 





A A 
a= Fgh? 7 = Ogee 
in terms of which 
A =o(2z)'?, A, = o(22,)'". (8.45) 


Using (7.59) and (7.60), we see that the one and two-dimensional 
distributions of the random variables 2 and z, are 








w(z) =e*, (8.46) 
wat re 29 Ve zt 
22) = Tao ( io za) exp | ~ Toe (8.47) 


Tt can be shown? that #(z, z,) has the series expansion 
=e > L,(2)L,(2,) 2 8.4 
w(z, 2,) = &- >> (2) Lq( 2x) Gif (8.48) 
n=0 


[cf. formula (4.74)], involving the Laguerre polynomials 


a” Ng— 2 
L,(z) = @ qa ere Nig (8.49) 
which are orthogonal with weight e~* on the interval (0, ~). 
It follows from (8.45) and the formula (8.46) for the one- 
dimensional probability density that the univariate moments of A 
are given by the formula 


CAT) = 2 Por(atl) = 2"Mof [evra dz = QloT & 4 1) 
. (8.50) 
= 2rMe (5) 1, 


3 The expansion in question is a special case of the Hille-Hardy formula. 
See Bateman Manuscript Project, Higher Transcendental Functions, Vol. IT, 
McGraw-Hill Book Co., Inc. (1953), formula (20), p. 189. 
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where we have used the familiar integral representation of the 
gamma function (factorial). To calculate the bivariate moments, 
we use the expansion (8.48) which implies that 


CATAN) = DUOtNRgrt (zt BZS/A = reanerts Y) Renlsn ae » (8.51) 
where h,,, denotes the quantity 
hen = J at e-tL,(2) dz. (8.52) 
0 


Substituting (8.49) into (8.52) and integrating by parts » times, we 
find that 


hoy = (—I53G — 1) G2 +1) re vP de = (—1)" a. 


(8.53) 


If r is even, then the quantities 4,, vanish for x > r/2. Therefore, 
the expansion (8.51) contains only a finite number of terms, 
provided that at least one of the numbers 7 and s is even. According 
to (8.53), when r = 2 we have 


Nog = 1, hy = —1, hog = hy =... = 0. (8.54) 
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Substituting (8.54) and (8.55) into (8.51), we obtain 
<A2A2) = 4o%(1 + 0%), 
(AA) = (A?A) = 0? Vin + 30) , (8.56) 


wT 1 
<AAY = SeA{ +408 + 08 + aor +) 


Then, using (8.44), we arrive at the final result: 


DW= a, + ao AS + 2a," , 
i 1 1 I 
Cy = 4) (¥, = a)> =F apo(L +50" + Ot + 55208 +.) 


+ daja,o?-VIa(1 + 40%) + daor(l +0%), 
(8.57) 
y(n) = 3 ator(Q? + 7g O* + Gg 0% + --) 


+ Vie aia,o*Q? + dazoAQ? . 


3. Moment Functions 
for Piecewise Linear Transfer Characteristics. 
The Direct Method 


The moment functions of random signals subjected to piecewise 
linear transformations can be calculated by two intimately related 
methods, one involving the use of series expansions of the two- 
dimensional probability density (the direct method) and the other 
involving the characteristic function and the evaluation of integrals 
(Rice’s method), In the next two sections, we shall use typical 
examples to illustrate the gist of these two methods, in keeping 
with our policy of deriving quantitative results of practical interest. 


3.1. Gaussian input processes. First, we calculate the mean value 
<n and the correlation function k,(7) of the process y(t) obtained at 
the output of a nonlinear device with a piecewise linear transfer 
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characteristic g(f), of the kind shown in Figure 6, when the 
input is a stationary Gaussian process €(¢) with zero mean and 
correlation function 


R(r) = o R(r). 
In this case, the one-dimensional probability density is 
wf) = 1 F (6) ; (8.58) 
where 
1 oft oa 
Fa) = Fe | _ereey, 


and the two-dimensional probability density w(f,&,) can be 
conveniently written in the form (3.14). 

Thus, we assume that the function g(é) is continuous and 
piecewise linear, and that its derivative g’(£) is discontinuous at the 
points ¢,, ce, ..., ¢, (see Figure 6). This means that 


(8) = Dba —c). 8.59 
e'(8) x 15(E — &4) ( ) 

Then, the mean value 
ay = fadoutenae =! teyr(t) a (8.60) 


can be written in the form 


Gy = Jim f ce ae) ar(é) : (8.61) 
Integrating by parts, we obtain 
[i aor) =atenr(#)-f eere)ja. 6.62) 
We now introduce the function 


Fax) = f ” F(x) de = xF(x) + F(x), (8.63) 
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which has the properties 

d 

BPO) = Fe), FM —w) = 0. 


Using these properties, and carrying out a second integration by 
parts in (8.62), we obtain 


[i oar) = ten F(t) — of" e@ar-n(8) (8.64) 
= (6) F (£) —og'(é;) pen.) 4 ms ie by e'(é) rew(é £) dé. 


According to (8.61), the mean value <> can be obtained by taking 
the limit of (8.64) as ¢,;-> 0. Since, as £5 >, F(é,/o) rapidly 
approaches 1, while F‘-"(€,/o) approaches ¢,/o [cf. (8.63)], it 
follows that 


o> = dim [olé) — Ge'Gal +o [err ae, 


§,90 


g(é) 





Fig. 6. A piecewise linear transfer characteristic and its derivatives. 
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or 
<> = firm [e(@) — ge" +e DoF), (8.65) 
tel 


where we have used (8.59), Moreover, from (3.14) we find that the 
second moment of (zt) can be written as a series of the form 


mo = > iB i _sermn 8) ag FO. : 


whose first term is just (4>*, according to (8.60). Subtracting this 
first term from <y7,>, we obtain the correlation function 


A,(r) = Sic: eg) Fon(2 £) det eile (8.66) 


To evaluate each of the integrals in (8.66), we again integrate by 
parts twice. For x = 1, we obtain first 


[or Qt=--[ ore, 
and then 
[sor = -e@re) +f" e@rG) a. 667 


For > |, the integrals can be evaluated even more simply, since 
the corresponding integrated terms vanish at +. Finally, substi- 
tuting (8.59) into (8.67), we find that 


[sors Em ogo) +0 DoF 1), 


ie Aeren(’) . = 0 aren(-) (n>1). 
-2 = 


We must now substitute the expressions just obtained into 
formula (8.66). A variety of special cases are possible, as shown in 
Figure 7, If g(£) consists of a single line segment, so that r = 0, 
b, = 0, then only the first term remains in (8.66). This case 
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corresponds to amplification of the signal | g’ | times, together with 
a possible change in the d-c component of the signal. Next, 
suppose that g(€) consists of two line segments, joined at the point 
& = ¢,, where g’ = a to the left of c, and g’ = 8 to the right of 
¢, (see Figure 7a), Then 


g(y)=8, bb =B-a, 


and we have 











Ao) = [2-H — Ay F(2)] RO)+6 — 9) S [Fon yP SO. 
(8.68) 
gté) 
$ 
ce) C, g 
tan $,=@ tang=S 
(a) tang,=B (b) 
g(€) g(€) 
¢ 
Cy : 0 ¢, é 
(c) tang=S (d) 


Fig. 7. Typical piecewise linear transfer characteristics, 
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In particular, if the “break point” is centrally located, i.e., c, = 0, 
then 
2 2. 2 R* 
f(r) = F(a + BF RG) + 08 — a)? S [Fev] LO 
at ™ (8.69) 
= 5 (a4 BF RG) + (Bo) (Rare sin RE-VT=RE — 1). 


By examining the case 


ae ee for £>4, 
for <q, 


aé) = (8.70) 


(see Figure 7b) where 
a=0, p=S, 


we can consider ‘‘peaks’’ of the random function &(t) and we can 
calculate the degree of correlation between them. In this case, we 
have 


k,(r) = stot|FA(— “L) Re) +3 4 [Fe-()] t Rr). (8.71) 


In Figure 8, we show the results of calculations of the correlation 
coefficient 


fale) 
Ril”) = 30) 





based on this formula, when terms up to = 7 are retained in the 
series and when the correlation function of the input process is 
R(r) = e—** (shown as the dashed curve). Curves corresponding to 
five values of the threshold y = c,/o are plotted. 

As an example of a transfer characteristic with two break points, 
we consider the asymmetric limiter 


Sc, for §>c,, 
n= el) = {SE for «<é<q, (8.72) 
Sey for <q 


(see Figure 7c), where 
g()=0, b=S, b=—S. 
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In this case, the correlation function is 
ky(7) = Sto? pauses 2) aEea 4 4)/ =o RG) (8.73) 
and, according to ‘een and (8.63), the mean value is 


Gy = Sq, +S aF(2) a eh() 4: oF (2) 2 oF ()] 
(8.74) 


To find the variance of the output signal in each of the examples 
just discussed, we can set r= 0, R= 1 in formulas (8.68), 
(8.69), (8.71) and (8.73); this gives the variance in the form of an 
infinite series. However, we note that it is simpler to find the 
variance in the ordinary way, by using the one-dimensional prob- 
ability density w(f) to calculate <n?) — <y>*, since this will give 
the result in finite terms. 





2 vat 


Fig. 8. The correlation coefficient R,(7) corresponding to (8.7%), 
when R(r) = 72", 


The method used above, which is based on integration by parts, 
is also effective for calculating higher moments., The method can 
also be generalized to the case of piecewise continuous transfer 
characteristics of more general form, e.g., characteristics with 
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jump discontinuities or characteristics made up of parabolic 
pieces, For example, if the transfer characteristic is 


q for £>q, 


o= (0 Fees (8.75a) 


(see Figure 7d), then a single integration by parts is enough, while if 


_ (P(E —GP for E>q, 
a= {9 dee ae (8.75b) 


then three integrations by parts are needed. The resulting expres- 
sions for the correlation function are 


¢: 


© | 2 
k(t) = Pa [Fo(2)] Rr) (8.76a) 
and 
Cc) 3 
k,(7) = 4pPot La [Fn->(— +4) R%z), (8.76b) 
respectively. 
3.2. Rayleigh input processes. We now consider two cases where 


the input signal is a Rayleigh process and the transfer charac- 
teristic is piecewise linear, 


Example 1. Suppose the input process [whose one-dimensional 
probability density is the Rayleigh distribution (7.59)] undergoes 
the transformation 


_\q for A>A,, 
Ve 3 for A<A,. (6.77) 
Introducing the process 2(t) = A?(t)/20?, as before, we have 
_|¢ for z>2, 
az, i for z<2%, (8.78) 


where 
AR 
‘9° sa: 
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Then, using the distributions (8.47) and (8.48) of the process 2(t), 
we find that 


mM =49 c e-* dz = ge-*o, (8.79) 
oO Gn 00 = 2 
om) = ¢ 2 Gap [f _baae dz] . (8.80) 


To evaluate the integrals appearing in (8.80), we use the definition 
of the Laguerre polynomials, obtaining 








a dn 
L,(z) e~? dz = — (ate > 
J a" dg ) smte 

which can be written as 

[Beet ds = 2b (e,) %, (8.81) 

30 
in terms of the associated Laguerre polynomials 

L(2) = 2-het & (z+1e-2) (8.82) 
nm dz . . 


Therefore, according to (8.79) and (8.80), the output process, which 
is actually a sequence of pulses, has the correlation function 


he(r) = ge at > ~ POLO (a)F. (8.83) 
n=l 


For reference, we write down the first few of the polynomials (8.82): 
L%2)=1, LM%(2)=2—2, Lz) = 6 —6242?, 
(8.84) 
Li0(2) = 2A — 362 + 122? — 29, ... 
Using (8.84) and substituting 2) = Ag/20? into (8.83), we obtain 


A? 2 
aa) Ot4(1 - Sh a) Jot]. 
(8.85) 








(7) = ? a 
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We could calculate the variance of 7(t) by setting 7 = 0,Q = 1 
in (8.85). However, it is simpler to make direct use of the one- 
dimensional probability density, this leads immediately to the 
formulas 

CP) = Pero, 
Dy = £,(0) = g’e-*o(e* — 1). 


Dividing (8.83) or (8.85) by the variance Dy, we find that the 
correlation coefficient is 


Rr) =O — BlOr%e) + BOM) + BON) +..J, (8.87) 
(0) 


(8.86) 


where 


AS cas 
B= Fox (eAtieet -Iy, 





eet Ag Aj \* 
B= (1-3 + Hu) - 


In Figure 9, we show the results of calculations based on formula 


0.5 





fe) I 2 vat 
Fig. 9. The correlation coefficient R,(7) given by (8.87), when Q(z) = eur, 
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(8.87), where only the indicated terms of the series have been 
retained, These calculations were made for the case where 

Q(r) =n 
(shown as the dashed curve). The quantity 

= fo _ 

= Go Go 


specifies the operating level of the limiter, and we have plotted the 
curves corresponding to the values y = —1,0,1,2 and 3. It 
should be noted that 


for a Rayleigh process, 


Example 2. Suppose the input Rayleigh process A(t) under- 
goes the transformation 


= {4 — A,) for A>Ap, 


for A<A,, (8.88) 


which “cuts off the envelope peaks’ exceeding the threshold 
A = Ay. Using the univariate Rayleigh distribution (7.59), 
we obtain the mean values 


(> = So 20 F(—a), 
CoP) = 2S%o%fem? — 4/2 aF(— 
and the variance 
Dp = So*[2e-2? — 2 V2 aF(—a) — 2nF{—a)], (8.90) 


where a = Ag/o, 

To find the correlation function k,(r), we use the expansion 
(8.48) of the bivariate distribution. First, we note that the trans- 
formation (8.88) is equivalent to the transformation 


(8.89) 


ae for z>2%, 
n= V2S0 for 4265, (8.91) 
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where, as usual, 


se Ve a Ao 
BS Bet? 70 Dok 


After subtracting <7>? from (8.48), we have 





iJ Ld 

h,(r) = 2S%o? > Ta Qr"(r), (8.92) 

where 
= [ (at — 2) L(e) e* de. 
29 

Next, we integrate this last expression by parts, obtaining 

Whee J 21 (2) e# dzi?, (8.93) 

20 

where we have used the relation 


L,(2) €-? dz = d[2L™,(z) e~*], 


which is equivalent to (8.81). 
Introducing the parameter p», and then differentiating with 
respect to 2, we can write (8.93) in the form 


hy = a x) [len dali2 l., 7 (8.94) 


Then, setting z = A®?/2o0® = a®/2, we express the integral in 
(8.94) in terms of the probability integral (see p. 42): 


fre en dain = 7 5 J. ewe da = NE F(-« Vi 
% 
It follows that 


ia = Vig O(— 3) Lee-ava] | 69) 
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For computational purposes, it is more convenient to use the 
somewhat different formula 


hy = Vie a [eam F(—« vn)| ie (8.96) 
which can be verified by using the relations 


—2L0(2) = Dy (r ) (a — 1) (n — 2)... (—z)', 


on OF, ny Or"? OIF, 
2 (7) = 2) be oe 5-2 () Qun-* ae 





an 
= ou (u"F) , 


where 
Fy= ai Vu). 
The first few coefficients h,, are 
hy = — Ma) + aF @), 
hy = — A UR(—«) + oP @) + FF), (8.97) 
its a "TF(—a) + af (2) + 2o8F'(a) + $2 F'"(a)] 


If we retain only the coefficients (8.97) in our calculations, 
setting all the others equal to zero, then the correlation coefficient 
of the process (2) [which is just the random sequence of peaks of 
the envelope A(t) exceeding the threshold Ag] is given by the 
expression 


R,(r) = Bo (wor +4 +5 /Ot +O!) (8.98) 


“Cf. I, M. Ryshik and I. S. Gradstein, op. cit., formula (7,140.1), p. 381. 
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where 
B= el? — 27 aF(—a) — nF —a). 


(Cf. formula (8.90).] In Figure 10, we show how R,(r7) depends on 


0.5 





° 35 0.7 105 Vat 


Fig. 10. The correlation coefficient R,{7) given by (8.98), when Q(7) = eo, 


the value of the threshold 


y= 40 ~ 1.25 = -1,0,1,2,3, 


o 


in the case where 


Ol) = eo. 


It is clear from the figure that the correlation coefficient of the 
peaks can differ appreciably from the function Q(7), shown as the 
dashed curve in Figure 10.5 In fact, R(r) “spreads out” for small 
thresholds, and then “shrinks,” starting from some value of 
y +0. 


7 5 Q(z) can be regarded as a kind of envelope correlation coefficient, but, 
as shown by (8.56), O(7) # <AA,> — <A)’. 
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4. Moment Functions for Exponential Transfer Characteristics. 
Rice’s Method 


41. The moment functions of the output 
a(t) = her” (8.99) 


of a nonlinear device with an exponential transfer characteristic can 
be expressed in terms of the characteristic functions of the input 
signal £(t), This fact can be exploited in cases where the charac- 
teristic functions are known, e.g., when &(t) is a Gaussian process. 
By averaging (8.99), we obtain 

<a> = 4O(—1a) , (8.100) 

<a> = h?@,{—ia, —ia) , 
where 
O,(u) = <e“®>, 


(8.101) 
O,(uy, ua) = <ermrttteatey 


are the one and two-dimensional characteristic functions. If &t) 
is a stationary Gaussian process with zero mean value, then as we 
know [cf. (3.1)] 

@,(u) = exp {— for}, 


@,(u,, uy) = exp {~ $ o°[u? + 2R(r) uu, + uz]} . 


Then, we can use (8.100) to find expressions for <n), <7,> and 
the output correlation function 


(8.102) 


A(t) = Hestetfeatotaesy _ 1], (8.103) 


When the transfer characteristic of the nonlinear device is not 
exponential, we can still use characteristic functions to calculate the 
moments of the output process. To do so, we use Rtce’s method, the 
basic idea of which is to represent the transfer characteristic of the 
nonlinear device by a contour integral 


1 = a8) =z [Pua e d0, (8.104) 
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where L is a suitably chosen contour of integration in the complex 
Q-plane. If the function g(é) vanishes at infinity (€ = +0), we 
can choose (8.104) to be the Fourier transform; then 


FQ) = [ ip AE) et de, (8.105) 


and we need only consider real values of Q. If g(Q) vanishes for 
€ < 0 (or at least for € less than some fixed number), we can apply 
the theory of the Laplace transform, by regarding (8.104) as the 
inversion integral 


CHIDO 
8) = 35 Lf" oF (P) dp 


of the transform 
Fp) = [ert 6)dé (p= i). 


In this case, if g(€) grows no faster than some power of é, as 
€ —>» + o, we can choose the contour of integration in (8.104) to be 
the real axis (c = 0), with a downward indentation at the origin. 
(The indentation is to the right inthe complex plane corresponding 
to p = iQ.) If g(é) goes to infinity in both directions (i.e., as 
€— +c), then it is appropriate to use the theory of the two-sided 
(bilateral) Laplace transform. 

Averaging first (8.104), and then the product of two expressions 
of the form (8.104) evaluated at the times ¢ and ¢ + 7, we can 
express the moments <7) and (77, in terms of the characteristic 
functions of the input process, which are assumed to be known: 





a> = xe | FOR) exp (128)> a2, 
(8.106) 
Coom> = goa J, [, FUR)FUD) Comp (128 + :2,60)}> dO AO, . 


For example, if &(¢) is a stationary Gaussian process, then, according 
to cae we have 


Sm) = aa af, J FU) PUA) exp {— fo'(O+2R20, +.) do dO, 
(8.107) 
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If this integral cannot be evaluated explicitly, we substitute the 
expansion 


o-oo, == S (IP (<I oemRnanan (8.108) 


nad 


into (8.107), obtaining 
an) = De 18 mR), (8.109) 
net ag 
where the coefficients 


Iy = 5 =f, F(iQ) e~#*9*? Q dQ (8.110) 


are much easier to compute. Then, using the first of the formulas 
(8.106), we subtract (7)? from (8.110), which gives the correlation 
function 


St 
Ri 2 eee (8.111) 


Example 1. To illustrate Rice’s method, we find the correlation 
function k,(r) corresponding to the nonlinear transformation 


AvV1 —®(t) for [El < 
at) = {9 for {él> 


In this case, we can simply take the Fourier transform (8.105), 
obtaining 


; (8.112) 


1 
F(iQ) = Af eit /T — Bde, 
“1 
where the integral is readily expressed in terms of a Bessel function,® 
he, 


F(iQ) = 7A 1) | (8.113) 


*T. M. Ryshik and I. S. Gradstein, op. cit., formula (6,413.3), p. 312. 
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and the inverse transform is just 
A [? 2 
i = 4 f° AD est an. (8.114) 


For Gaussian noise with the characteristic function (8.102), the 
desired correlation function is given by (8.111), where the coeffi- 
cients (8.110) are 


hn = 4b Am i JMO) e242 Q-1 GO (8.115) 


1 
2 
Since J,(Q) is odd, the integral in (8.110) vanishes unless 7 is 
even, i.€., # = 2m. Introducing the notation 


foe (Io) = ot [ Jy(@) est" Oe dO = [" J, (xfo) eH xt de, 
0 0 
(8.116) 
we have 
0 for odd 2, 


ee er ati 
as Bin (Ifo) for n = 2m. 





on 
Then, substituting (8.117) into (8.111), we obtain 
2 | a 
R,(r) = 4D Gnyy Siam (Ifo) R°"(r) . 
The coefficients g,, can be expressed in terms of the confluent 


hypergeometric function by using formula (4.434.2) of the hand- 
book just quoted,’ ie., 


Bole) = | Solar) el wet de @.118) 
= raven TUE DEL crew FPA 4 tsp +15 =), 
wherep+q>0. 


7], M, Ryshik and I. S. Gradstein, op. cét., p. 237. 
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4.2. Next, we consider the case where the input process is the 
sum of a Gaussian process ¢(¢) with the characteristic functions 
(8.102) and a “sine wave” 

s(t) = Ecos (wot + y) = Ecos®, 


with fixed amplitude EZ and completely random initial phase ¢, 
(cf. p. 198) which is statistically independent of é(t). Then the 
characteristic function of the total signal 
xt) = &t) + s(t) (8.119) 
is just the product of the characteristic functions of the processes 
&(t) and s(t): 
deine) — Cet28y CeifE Coa, = e002 JAE) i (8.120) 
Here, to calculate <e*), we have averaged over the random initial 
phase go, using the integral representation of the Bessel function 


Jz). Similarly, for the two-dimensional characteristic function, 
we have 


(ei9z4i0,2,) 
= Cexp {226 + 12,£,}) Cexp {iE(2 cos P + 2, cos B,)}> 


= expj— ; AQ + 2RQQ, + 2) J (EVI F ION, cos ar + 


(8.121) 
[cf. (7.82)]. 

To determine the mean values <7) and <yn,>, we substitute the 
characteristic functions (8.120) and (8.121) [instead of the functions 
(8.102)] into (8.106), In calculating <77,), it is convenient to use the 
familiar addition theorem for Bessel functions® 


JE VET 10D, cosa FG) = b} (—1)* ex], BQ) J({EQ,)cos kur, 
= 
(8.122) 


*I. M. Ryshik and I. $. Gradstein, op. cit., formula (6.541.1), p. 335. 
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where 

eg_il, € =e =. = 2. 
We also use the expansion (8.108) for the quantity exp (—o?RQQ,) 
appearing in (8.121). After making all these substitutions, we 
obtain the following double series for the moment ¢y7,> of the 
output signal: 


com> = 3 SAF J, Fea) Ja) eater ao 
es (8.123) 





(=) 
n\ 


x €,R” cos kart . 

To obtain the correlation function &,(r), we have to subtract from 
(8,123) the term corresponding to n = 0, k = 0, which equals 
«ny. 


Example 2. Suppose the process (8.119) undergoes the 
transformation 
fc >0, 
er ey 29 
Then we have 


eit x de = — 1 (8.125) 


FUiQ) = J ar 


o 
where the contour of integration L coincides with the real axis 
Im Q = 0 when the integral has no singularity at the origin 2 = 0, 
and otherwise has a downward indentation avoiding the singularity 
at the origin, According to (8.123) and (8.125), we have 


Rr) = oa y [of JA EQ) €-08712 Qn-e ao} 
ise. ce 
(8.126) 
(—l"* 


m nl 





e,R(7) cos kwor . 


Only two integrands in this sum have a pole at the origin, i.c., those 
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corresponding to n = 1, k = 0 and n= 0, k = 1. Since these 
integrands are odd functions of 9, it is easily seen that 
f JA EQ) eotaye 0 =in, 
ie (8.127) 
1 dQ E 
= o82t/2 i, 
a}, BEM 0 oe = Hit 


All the other integrals in (8.126) can be expressed in terms of the 
functions (8.118). As a result, (8.126) reduces to 


cal E? 
k,(7) cay Rr) + z COS WoT 
(8.128) 


el 
ae 2 sg ll + (HD expt n-1( Efe) RM(r) cos Regt « 
nik=d 


ntk>l 
In particular, (8.128) is valid for a narrow-band random process 
lt) = A(t) 00s (wt + 9) 
with the correlation function 
R(t) = r(x) cS wor , (8.129) 


where A(t), p(t) and r(r) are slowly varying functions, e.g., 
r'/r < wo, In this case, the output signal 


a(t) = g[A cos (wot + p) + Ecos (wot + 9o)] = g[B cos (wot + ¥)] 
i (8.130) 


has spectral components lying in narrow bands near the frequencies 
lw, (J = 0, 1,2, ...), and the correlation function (8.128) can be 
‘written as a series 


fog 7) = Ror) + F(t) COS wogr 4 yr) COS 2wrgr + on, (8.131) 


involving the slowly varying functions ko(r), ky(7), Ro(7), « 
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Of particular interest is the first function &,(r), obtained by 


averaging k,(7): 


fn 


hr) = 52 k,{r') dr’. (8.132) 


t 


We now show that in the case of narrow-band signals, the averaged 
correlation function (8.132) is approximately equal to the correlation 
function K[%, 7,] of the time-averaged signal 


£4.(27/arg) 


Ae) = Ff ate ae = % [" alt-+x) dx, (8.133) 
where Ty = 27/wy. According to (8.133), we have 
Kidda f [arte —y)dedy. 6.134) 
Blo Jo 
If we substitute (8.131) into (8.134), using the Taylor’s series 
hr ts— 9 =< LHR = 012.0), 


then, since 


T. pT 
Jo [Ade 008 hung +» — y) dx dy = 0 for k>0 
o 0 


and 
7 a 
| [0° fo" — 39" c0s kane + 2 — y) de dy 
0 0 
Te pto _ 275? 
(7 ii lecpdedr= Chery: 
we find that 


KL, 3] = A(x) + ORT, + iT +o) « 
It follows that 
R(t) ~ Kh, i) » 


as asserted, since the narrow-band condition implies k,T, <k,. 
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It is most convenient to carry out the averaging (8.132) of the 
expression (8.123) before the expansion (8.108) in powers of R is 
made in (8.121). Thus, expanding only the Bessel function factor in 
(8.121), and using (8.129), we find that 


(ei22+i9,2,) 


= eta QOH /2 D2 (=1)¥ eeJic (EQ) Jy EQx) €- 77992700807 COS Rergr. 
ken 
(8.135) 


We then carry out an average of the type (8.132) with respect to 
7, regarding r(r) as constant; as a result, (8.135) becomes 


en FB 27)/8 > (-1¥ & Je(E2) J(EQ,) L(—o722,), (8.136) 
kno 


where J,(z) is the modified Bessel function of the first kind, of 
order &.° Since 
a. 1 1 antk 
I —0%2,) = 2 aie EBT (- 5 o122,) 


by analogy with (8.123) we have 


<i) = Ps [= J FG) JAEQ) e028 Qe da] 
- (8.137) 


2ntk 


ae tar G7] 


Subtracting <7>? = <n>%, the square of the mean value, from 
(8.137), we obtain the correlation function 





k{7) = ee 
FQ) Jf BQ) e-s20"P Qin aa] ae oa “ ii [5 x=) nr yn 
jitkoo (8.138) 


since <7)? is just the term in (8.137) with k = 0,” = 0. 


= I. M. Ryshik and I. S. Gradstein, op. cit., formula (6.443), p. 317. 
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If the nonlinear transformation g(x) has the form (8.124), then the 
d-c component (8.133) is proportional to the total amplitude, i.e., 


F=f, Bowe tae =F, (8.139) 
ry W 


and hence (8.138) can be used to find the correlation function of the 
amplitude B(t). Substituting (8.125) into (8.138), we find that 


Re{r) = wk,(r) 


a 
~ it 2, 
n+k>0 


[om Head iu ‘a J EQ) €- 979712 Qantk-2 aa} 
-% 


& 1 antk 
* iin + Ay! br) , 
so that, according to (8.118), 


€, 
ls * efi FR TRE Lanse (Ele) (7). (8.140) 


Mtk>0 


CHAPTER 9 


Nonlinear Transformations with 
Memory. Detection of Random Signals 


To illustrate the basic idea of the methods used to study what 
happens when random signals undergo nonlinear transformations 
with memory, we shall consider the example of the diode detector, 
whose action is described by a first-order nonlinear differential 
equation. By the diode detector, we mean the device shown in 
Figure 5, obtained by connecting a diode, i.e.,a nonlinear element 
with current-voltage characteristic J = F(V), to a parallel RC- 
circuit.! Suppose we want to find the statistical characteristics of the 
voltage (t) across the RC-circuit, when the input to the detector is a 
random signal ¢(¢) with known properties. The differential equation 
for the diode detector has the form 


1 1 
at Ror = GFE -1) (9.1) 


[cf. (8.3)]. A rigorous solution of this nonlinear differential equation 
for typical functions F approximating actual current-voltage 
characteristics is impossible even in the case where the function é(t) 
in the right-hand side of (9.1) is an ordinary (non-constant) 
function of time, rather than a random function. Thus, solving (9.1) 
is certainly all the more unfeasible when &(t) is random, and we 


1 The ordinary diode rectifier, with a purely resistive load, is a simpler 
system, since there is no need to take the reaction of the load into account. 
fn this case, the problem reduces to the study of a zero-memory trans- 
formation of the input process, like those studied in the previous chapter. 
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must have recourse to approximate methods of solution. The choice 
of one method as opposed to another depends on the form of the 
function F(V), on the system parameters, and also on the relation 
between the correlation time 7,,, of the input process and the 
time constant of the detector (equal to RC). We can distinguish 
five special cases, each of which corresponds to a different way of 
handling the problem: 


1. If the inequality | 
Teor > RC (9.2) 


holds, we can study the problem in the quasi-static approximation. 
In this case, as a first approximation we can neglect the time 
derivative 7 in equation (9.1), and then the problem reduces to the 
zero-memory nonlinear transformation 


7 = RME—7)- 


2. Suppose the input process &(¢) is narrow-band, i.e., corre- 
sponds to oscillations of frequency w, with slowly varying amplitude 
and phase, so that €(t) has a large correlation time 7,,,. If the 
relation 


Top > RCD = (9.3) 


holds, then we can apply the envelope method, which is a variant 
of the quasi-static method. This case includes the detector stages 
used in many radio receivers. 


3. In many cases, it is appropriate to use the method of ‘‘small 
nonlinearity,” i.e., the problem is regarded as linear in the first 
approximation, and the nonlinear effects are only taken into 
account in subsequent approximations. This method is applicable 
when the nonlinear term (or terms) in the original equation is small, 
or when the equation can be linearized with respect to deviations 
from some suitably chosen zeroth approximation. The application 
of this method is not limited to the case where there is some special 
relation between the time constant of the system and the correlation 
time, and in fact the method can even be applied when 7,., ~ RC. 
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4. If the time constant is small, which means that 
q 


Teor < RC, (9.4) 


i 

then &(t) can be regarded as a Markov process, and we can make use 
of the Fokker-Planck equation. In particular, this case may arise in 
making measurements, when we want to ascertain the mean value 
<n) of the output signal (characterizing the reading of the measuring 
device) and the variance Dy (characterizing the measurement 
esrgr). Then, it is sufficient to know the one-dimensional stationary 
distribution, which is a solution of the Fokker-Planck equation. 


5. When we are dealing with intermediate values of the correla- 
tion time, Le., 
Teor ~ RC, 


and when the signals are strong, the problem becomes very 
difficult. If the input signal £(¢) is assumed to be a Markov process, 
the output voltage will be a component of a multidimensional 
Markov process, and some results can be obtained by using the 
multidimensional Fokker-Planck equation. 

We now study these various methods in turn. 


1. Detection of Narrow-Band Processes. The Envelope Method 


If the input process is slowly varying (tor S> RC), then, as 
already noted, we can neglect the derivative 4 in equation (9.1), 
obtaining 

} 
4 9 = RF(E—7)- (9.5) 
Solving (9.5) for 7 in terms of ~ (for example, graphically), we 
obtain a zero-memory transformation 


alt) = alé(e)] - (9.6) 


Me-hods for analyzing nonlinear transformations of this kind were 
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presented in Chap. 8. It is sometimes convenient to represent the 
function (9.6) as a polynomial, e.g., of the form 


a(é) = aE + 2,8, 


BE) = 428" — ays. 


Then, the coefficients a,, a, and a, are chosen numerically in such 
a way that the polynomials give a satisfactory approximation over 
the important part of the transfer characteristic. The statistical 
characteristics of the output signal for these two polynomial 
transfer characteristics were calculated in Examples 1 and 2 of 
Chap. 8, Sec. 2. 


Linear Terminal 


Detector 1-f gm- i 
plifter Device 





Fig. 11. Simplified block diagram of a typical radio receiver. 


We now examine in detail a variant of the quasi-static method 
which is suitable in the case where the input signal is a narrow- 
band process. Consider a typical radio receiver (shown schema- 
tically in Figure 11), consisting of a linear selective intermediate- 
frequency (i-f) amplifier, a detector stage, a linear low-frequency 
(l-f) amplifier, and a terminal device. Usually, the i-f am- 
plifier is a frequency-selective four-terminal device, i.e., the 
inequality Jw <we usually holds, where wo is some “center” 
(angular) frequency, and 4w is the width of the pass band of the 
amplifier. Thus, if the input to the i-f amplifier is a broad-band 
stationary process, the output 7(t) is a narrow-band process, 
which usually has a correlation function of the form 


R(t) = 7(r) cos wor, 


where 7(r) is slowly varying compared to cos wor. [In formula (7.4), 
the function s(r) vanishes because of the symmetry of the frequency 
response of the i-f amplifier about the center frequency w.] 
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Like every narrow-band process, é(#) can be written as an 
oscillation 
E(t) = Ald) cos [wet + oD], (9.7) 


with slowly varying amplitude and phase (7.55). If in addition to the 
process £(t), the input to the detector contains a sine wave E cos wot, 
then the total signal &t) + Ecos wot can also be written as an 
oscillation of the form (7.83), with a different amplitude B(t) and 
phase ¥(¢). Thus, the results to be obtained below can be extended 
to this case, if we simply replace the amplitude A(t) by B(t). 

S The function of any detector is to ‘“‘extract” the low-frequency 
‘modulation from its input in as effective a manner as possible, and 
‘hence the output voltage across the RC-circuit must manage to 
“follow” the amplitude changes. Since the amplitude varies with 
the time constant 7,,,, this means that the condition 


Teor > RC (9.8) 


must hold. On the other hand, the detector must exert a strong 
smoothing action on the radio-frequency (r-f) oscillations, which 
means that the condition 


RC>2 (9.9) 
Wo 


must hold. If these two conditions are satisfied, the study of the 

detection problem is greatly simplified, since then, as we shall see, 

the output voltage 7(t) is related to the amplitude A(z) [or B(t)] of 

the input oscillation by a zero-memory transformation. 
Substituting (9.7) into (9.1), we obtain 


1 1 
4+ RET = GFA cos (wot + @) — a - (9.10) 


Integrating (9.10) over a period from ¢ to t + (27/w), we find that 


» Om 
alt + —) — a(t) 
( in) (9.11) 
3 t+ (A/w,) 
= f 8 (an(t’) + RF[A(L') cos (wot’ + 9) — a(t’) at’ - 


t 
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However, because of (9.9), the function x(t) only manages to 
change very slightly during one period. Therefore, the difference 


aft + =) —~ alt) 


is almost equal to (27/w,)j, and 7(t’) can be regarded as constant 
in the integrand, i.e., 7(t’) ~ (t). The amplitude A and the phase » 
can also be regarded as constant during a period, so that we need 
only take into account the change of cos [wot’ + 9(t)]. Thus, (9.11) 
reduces to 

- 1 1 cq ptti24/~o) ; 

i+Re l= cm), FLA cos (wet’ + @) — 7] dt 
or 


: 1 1 1 7 
A+ ROI= Tig), FAW 08x — a dy. (9.12) 


The right-hand side is essentially a known function of A and n, 
obtained by integrating the current-voltage characteristic F(V) of 
the diode. 

As it stands, equation (9.12) is hard to solve in general form. 
However, the problem is greatly simplified by the condition (9.8) 
which requires that the amplitude vary slowly; in this case, we can 
neglect the derivative 7 in (9.12). In fact, just as in the case of A(t) 
itself, the only important spectral components of the process 7(t) 
lie in the frequency band 

Je | ~ 4a = =~. 
For these frequencies, the derivative 7,, = iwy, is much smaller 
than 7,/RC, because of (9.8). Therefore, the derivative 4 in (9.12) 
is much smaller than the second term 7/RC. If we neglect 4, 
(9.12) becomes 
R 

ae F(A cos x — 7) dy, (9.13) 
which specifies the zero-memory transformation leading from the 
amplitude A(t) to the output voltage 7(t). 
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We now use (9.13) to study three typical cases of diode detection, 
where the current-voltage characteristic F(V) is piecewise linear, 
quadratic and exponential, respectively. 


Example 1. In this case, usually called the linear detector, the 
current-voltage characteristic is 


1 
1=FV) = RY for V>0O, 


0 for V <0, 


(9.14) 


where R, is the internal resistance of the diode in its conducting 
state. Substituting (9.14) into (9.13), we have 


AR /Y 
os mR, , (98x — a) OD 
where 


y = are cos + . 
We now introduce the dimensionless quantity 
=. 
haz (9.16) 


which can be called the “envelope reproduction coefficient.” 
Then (9.15) becomes 


k= 





R arc cosk 
Ri, | (ex Ady, 


7. 
and carrying out the integration, we finally obtain 


wR 

= 9.17 

, VI —R — kare cosk On) 
If R/R; < 1, the envelope reproduction coefficient k is < 0.3, 

and then (9.17) implies that 

1R 


kw oR 
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However, such values of R/R; are not encountered in typical radio 
receivers. In fact, this ratio is of the order of 3 to 10 in radar and 
television receivers, and can equal several hundred in communi- 
cation receivers. In Figure 12, we plot the coefficient & for large 
values of R/R;. It is clear from the figure that the envelope repro- 
duction coefficient & grows monotonically with R/R;, and 
approaches | asymptotically. 





fe) 
\ 10 100 1000 RF 


Fig. 12. Dependence of the envelope reproduction coefficient k on the 
resistance ratio R/R, for the linear detector. 


The characteristic feature of the linear detector, as opposed to 
other kinds of detectors, is that its output voltage 7(¢) depends 
linearly on A(), and therefore it is appropriate to introduce the 
envelope reproduction coefficient (9.16), which does not depend on 
A. After calculating &, the statistical characteristics of the output 
signal 

a(t) = RA(t) (9.18) 


can be found very simply from the corresponding characteristics of 
A(t). In particular, we have 


<n) = RAD, Can) = RKAAD, 


where it is useful to recall that <A.A,) is given by the third of the 
formulas (8.56). 
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* Example 2. For the quadratic detector, the current-voltage 
characteristic is given by 
or V>O, 


_ (BV? fe 
FV) = |g for V <0, 


(9.19) 


where the coefficient 8 is a dimensionless constant. Using (9.13), 
we obtain 


_ RAB py a} 
1 = BO J (osx ~ A) ax, 
where 


y = arccos—, 


as before; again introducing k = 7/A, we have 


t RA are cos k 


k (cos x — hk)? dy . 


w 


Carrying out the integration, we obtain the following  tran- 
scendental equation, after making some elementary calculations: 


: 2nk 


PRA = Tayaccsk—akvinm (8) 


In this case, the envelope reproduction coefficient & no longer has 
its former meaning, but depends on A. 

Af BRA < 0.1, the inequality k <1 holds, and then, setting 
arc cos k 7/2, we find from (9.20) that 


kw dBRA, 7 46RAt_ (9.21) 


For large values of BRA, the coefficient & can be found by solving 
equation (9.20) numerically, In Figure 13, we plot the function 


k = f (BRA), (9.22) 
where f(x) is the inverse of the function 


x Spf 2) es nS 
(1 + 2k4) arc cosk — 3k V1 —~ 
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As the figure shows, the coefficient k grows monotonically with 
BRA, and approaches 1 asymptotically. 


1.2 











Fig. 13. Dependence of the coefficient k on the parameter BRA 
for the quadratic detector. 


It follows from (9.22) that the dependence of the output voltage 
on the amplitude can be written in the form 


BRy = BRA f (BRA) . (9.23) 


The function (9.23) can be approximated by a polynomial in some 
region or other, depending on the range of values of BRA most 
frequently encountered. For example, the quadratic polynomial 


BRy = 0.583(BRA) + 0.002(8RA)? 


plotted in Figure 14 gives a satisfactory approximation to (9.23) 


BRy, 
80 


60 
40 
20 


1°] 50 100 BRA 


Fig. 14. Approximate dependence of the output voltage on the 
amplitude for the quadratic detector. 
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in the range BRA = 0 to 100, but it is best to use this approxima- 
tion when BRA ~ 50, since it is not too accurate for BRA < 50. 
For amplitudes corresponding to a different range of BRA, one 
should use a polynomial with different coefficients. After choosing 
the appropriate polynomial, the statistical characteristics of the 
output signal can be calculated by using the method given in 
Example 2 of Chap. 8, Sec. 2. 


+ Example 3. Finally, we consider the exponential detector. In 
this case, the current-voltage characteristic has the exponential 
form 

T=F(V) = ie”, (9.24) 
where é, and a are constants, J is the current through the nonlinear 
element, and V is the voltage across it. The curve (9.24) can be used 
to approximate the current-voltage characteristics of actual diodes. 
with 

fy = 50 to 600 microamperes, 
a a =5 to 10 volts“, 


in the case of oxide-coated cathodes at normal working tem- 
peratures. 

If the input to the detector is the process (9.7), then, according 
to (9.13), the voltage across the RC-circuit is given by 


pe Me ices 2 
1 = Rip =— | exp {aA cos x —1)} dy = Ripe-°" (aA) 
o 
or 
ane*? = aRigl (aA) . (9.25) 

Here Ip(z) = Jo(éz) is the modified Bessel function of the first 
kind, of order zero, Equation (9.25) can be solved in various ways, 
desending on whether the quantity aRI,(a(A>) or the quantity 
a¢ A> is large or small compared to unity. 

lf a(A> <1, then 7 differs very slightly from the constant 
voitage 7) which appears in the absence of any input voltage, and 
corresponds to setting A = 0 in (9.25): 


Ange» = AaRiy. (9.26) 
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Using (9.26) and the expansions 
1 l 
Ifad) =1 +5 (0A) +g (aA +o, 
KE? = geo 4 (1 + xp) 70 (x — Xo) + (% — %)*, 
we find from (9.25) that 
= ao 2 
1 aT amy (9.2) 


‘Therefore, for small values a4 $0.1, the voltage across the 
detector load due to the presence of the input signal is proportional 
to the square of the envelope. In fact, we have quadratic detection 
quite generally for small input amplitudes, regardless of the form 
of the current-voltage characteristic F(V). This is related to the 
fact that the output signal is an even function of A, since it does 
not depend on the phase of the input signal, and does not change 
when the phase is shifted by 180°, i.e, when A is replaced by — A. 
Thus, only even powers of A appear in the power series expansion 
of 7 — 7%, aud hence for small values of A, only the first term of 
the series, proportional to A?, is important. 
‘The case where ARz, and aA are both large, i.e., 


aRigl (aA) > 1, (9.28) 


is of great practical interest. Then the envelope reproduction 
coefficient 7/A is large, of order unity. Using the condition (9.28), 
we can solve equation (9.25) by the method of successive approxi- 
mations. Writing (9.25) in the form 


x+Iinx=2z, 
where 
x=@n, x =In[aRiI(aA)}, 


we define the successive approximations by 


%, =2z—Inx,, (2 =1,2,..), 
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ne, 
m= 2, 
oa x =2z-—Inz, 


’ Inz 
%, = 2 —In(z —Inz)xz--Inz seem 


Confining ourselves to the second approximation, we have 
_ 


mh 1 iy 2Riol(aA) 


7 a In[aRigl(@A)] ° (9.29) 


Next, we use (9.29) to determine the coefficients a, and a, in the 
expression (8.42), choosing c to be (A), the mean value of the 
amplitude: 

7 = @ + a,(A — <A>) + aA — (AD). 
Equating the terms in this polynomial to the corresponding terms 
of the Taylor’s series expansion of (9.29), we obtain 


a =(1— al a, (9.30) 


where 


a, =InaRi, + Ini (aA), 9 = neat : 





As we increase the parameter a¢ A), 
37%, Fl, 


so that, according to (9.30), the coefficient a, > 1, while a, — 0. 
Tus, in this case, the transformation of the signal by the exponen- 
tial d stector approaches the ideal detector 


n7=a,t+A (a4 =a,—<A), (9.31) 


which reproduces the envelope exactly. 


’ 
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For an arbitrary current-voltage characteristic F(V), the more 
steeply the function F(V) grows compared to V/R, ie., the more 
valid the condition 


Ri<R, where R, = and V>0, (9.32) 


l 
OFieV 


the closer the detection is to being ideal. 


2. The Method of Small Nonlinearity 


We shall present the other methods of analyzing nonlinear 
transformations with memory by applying them to the special case 
where the current-voltage characteristic of the nonlinear element 
(diode) is an exponential function 


F(V) = iget”; (9.33) 


here, i, and a are the constants discussed earlier (p. 253). Then 
equation (9.1) becomes 


: 1 i 
4+ REI = 2 estan , (9.34) 


It is convenient to introduce the new variable 
x= em, (9.35) 
which satisfies the equation 
RCE + x Inx = atgRe™*. (9.36) 


The character of the process x(t) satisfying (9.36) is determined 
by the size of the parameter aigR, and also by the ratio of the 
correlation time 7,, of the process £(t) to the time constant RC. 
Depending on whether these parameters aigR and r,,,/RC are 
large or small, we can apply various special asymptotic methods for 
studying x(t). For example, the condition r,,, S> RC allows us to 
use the quasi-static method. On the other hand, if 7,,, << RC, we 
can use stochastic methods involving the Fokker-Planck equation. 
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Tiowever, neither of these methods is suitable if +,,, ~ RC. This 
compels us to have recourse to another method, which is appro- 
priate when the other parameter ai,R satisfies certain conditions. 
For the time being, we postpone the precise formulation of these 
conditions, and instead explain the method itself, which we call the 
method of small nonlinearity. In this method, we look for successive 
approximations to the solution x(t) of equation (9. 36). The zeroth 
approximation x,(t) is defined as the solution of the equation 


i RCky + x9 IN xg = aigR<e*) , (9.37) 


which is suggested by averaging (9.36). Assuming that £(t) is a 
stationary process, we are interested in a stationary process x(t) 
which began in the remote past. This means that a stationary 
‘value of x9 has been established, which, according to (9.37), 
satisfies the equation 


{ Xo In xq = atyR<e**) . (9.38) 

‘The average <e"*) is the value of the characteristic function 
<e) for u = —ia. If &(t) is a Gaussian process with zero mean, 
then 

Kelty = eoulf2, eat = eatotla, (9.39) 

and (9.38) becomes 
Xq IN x9 == aigheme*?2 , (9.40) 
Using (9.35), we find that 

No = i In xo 
satisfies the equation 
meet = igRes*ot/2 , (9.41) 


‘ Next, we derive a differential equation for the deviation 
, ; Bot — Xp. (9.42) 
Subtracting (9.37) from (9.36), we find that 


RCE 4- (xq + 2) In (xq + 2) — x9 IN xq = aigR(e*t — <e**). (9.43) 
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If the function x In x can be expanded in a Taylor’s series 


xing = agin xy + (Inxy + Dz tae at~ Gatti, (9.44) 
0 0 


then equation (9.43) becomes 
2 
RCE + (In x) +1) 2 = aigkt —-2- 4 2,—.., (9.45) 


where 
t= ett — Cent, (9.46) 


This equation contains the nonlinear terms 


Fe at afl t fed Se, 4) 
where 

f(x) =xInx. 
The use of the method of small nonlinearity is based on the 
assumption that these nonlinear terms have comparatively little 
effect, and can be neglected in a first approximation. Therefore, 
(9.45) leads to the following equation for the first approximation: 


RC&, + (In xo + 1) 2 = GigRE. (9.48) 


To clarify how higher-order approximations are constructed, we 
introduce an auxiliary small parameter «. Then, we write the 
solution (9.42) as an expansion 


B= 2 $ 2% + czy 4-0, (9.49) 


and we equip the nonlinear terms with appropriate small para- 
meters. ‘Thus, writing (9.45) in the form 


RC + (Ing + Ia = aight — £22 + Go - 
‘0 0 


substituting from (9.49), and equating terms of the same order in «, 
we find that the equation for the next approximation is 


RC2q + (In x + 1) 2 = — - ‘ (9.50) 
0 
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When it is inconvenient to make the ‘T'aylor’s scries expansion 
(9.47), or when the expansion does not exist, the whole nonlinear 
term (9.47) appearing in (9.45) can be multiplied by a small 
parameter. With this version of our method, the whole nontinear 
function 


Ff (% + 21) —f (%0) — f'(%0) 21 


will appear in the right-hand side of the equation for z,. Once the 
method of constructing the various approximations is clear, we 
can get rid of the small parameter, by setting it equal to I. 

The right-hand sides of (9.48) and (9.50) involve only known 
functions or functions already found in lower-order approxima- 
tions. Moreover, these equations are linear in the unknown 
functions, and can be solved without any particuler difficulty. 
Thus, the stationary solution of equation (9.48) is 


a= 2p ap|— Betta ntaeya. es 


According to the definition (9.46), the process {(¢) has mean value 
zero, and hence 
km) =0. (9.52) 


We now find the correlation function and the variance of the 
first approximation z,(t). First, we calculate the correlation function 
of the process {(t), assuming that €(#) is a stationary Gaus- 
sian process with mean value zero and correlation function 
R(r) = o*R(r). Because of (9.46), we have 


COE > = Centtabry — Cert, (9.53) 
The quantity <e***:) can be expressed in terms of the two- 
dimensional characteristic function, given by the familiar formula 


(8.102). Thus, evaluating @,(—ia, —ia) and using (9.39), we obtain 


<UL.) = estotfestetRnr — I}, (9.54) 
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According to (9.51), the correlation function z,z,, equals 
: ai, 2 pt tte 
Cataed = (SEY ff erteetn-oteeente (£4) L(t)» dty dle» (9.55) 
where 


_ Inxs +1 
i= RC 





Making the change of variables 
ay eee ahtth 
p=th—t—-T, s= 2 ; 
we can write (9.55) as 


iy 


cera) = (SAY fh ap f erametin ep 





After evaluating the integral with respect to s, this becomes 
Aa] @ 
Caren = (GE) ag Jee lewd (9.56) 


In particular, setting +s = 0, we find the variance 
5, \% 1 20 
= (BY gf mt a. 0.57) 
Using (9.54), we obtain 
we RY geet 1” (gatotree = 
amet Rel, CRP Derm. Cae 


To obtain more exact results, we can substitute (9.51) into the 
right-hand side of (9.50), and then calculate a higher-order 
approximation. 

The statistical characteristics of the output voltage 


7 = in (xe +2) 
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can now be found by expanding 7 in a Taylor's scries 
2? 


z 
gers (9.59) 


n= 1 + 
and then substituting the series (9.49) into (9.59). If we confine 
ourselves to results involving the first approximations (9.51), 
(9.56) and (9.57), we have 


1 
<> = 9 — Jaxt 4» 
i (9.60) 
<@ = 10) (ie — 70)? = Ra Aas 
0 


and in particular, 
1 
D7 = ae P i (9.61) 


To determine the right-hand sides of these expressions, we use 
formulas (9.56) to (9.58). 

We now analyze the conditions under which the above mcthod is 
applicable, for the most difficult case, i.c., where the process £(t) 
has a correlation time of the order of the filter time constant 
(Teor ~ RC). The method of small nonlinearity is applicable if the 
nonlinear terms in equation (9.45) have a small effect compared to 
the linear term (In x9 + 1)z. Choosing the first of the nonlinear 
terms, we obtain the condition 


[nay +1) 21> || 


or 
| z| <2xfIn x9 -- 1) (9.62) 


where for z we can take the first approximation (9.51). According 
to (9.38) and (9.40), the quantity x9(In x, + 1) differs only slightly 
from 

Xp In Xp = Ay Reel? ~ aig Re-¥0*!8 of L) , (9.63) 


where o(£) is the standard deviation of @, since according to (9.54), 


off) were", 
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On the other hand, using (9.51) we can obtain the estimate 


aiy RC 

as Cinm ti? (9.64) 
where the inequality sign corresponds to the case where the 
function ¢(t) changes rapidly, with time constant 


RC 
Teor S In 1° 
In the opposite case, where 7.., ~ RC, the quantity z, is chiefly 
determined by the factor e-" appearing in the integrand of 
(9.51), and then the sign ~ should be chosen in (9.64). 
Substituting (9.64) into (9.62), and taking into account the 


estimate (9.63) and the relation { ~ o(Z), we find that 


In x9 > e272, 


Multiplying both sides of this inequality by x) and using (9.40), 
we have 
aiRS> ry (9.65) 
or 
QigR In (aigR) S> xo In x9. 


Using (9.40) again, and dividing by ai,R, we write the small- 
nonlinearity condition in the form 


In (ai,R) > e227? (9.66) 


Thus, the method of small nonlinearity is applicable for large 
values of the parameter ai,R and for noise levels which are not too 
large. In the case of a small correlation time 7,,,, corresponding to 
the sign < in (9.64), we can replace (9.65) and (9.66) by less 
stringent conditions. Then, the region in which the method of 
small nonlinearity is applicable, which is essentially determined by 
the condition o(x) < x», becomes somewhat larger. 
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3. The Method of the Fokker-Planck Equation 


When the input process has a sufficiently small correlation 
time, i.e., one which is smaller than the other time constants of the 
system, we can analyze the process x(t) by using stochastic methods 
involving the Fokker-Planck equation. We shall illustrate this 
method by applying it to the previously considered problem of the 
exponential detector, described by equation (9.36), which we write 
in the form 

RCE 4 xin = aigReet® -) atyRE, (9.67) 


When +,,, is small, we can replace the random perturbation 
tm ett —- certs (9.68) 
in (9.67) by a Gaussian delta-correlated random process 2, which 


has mean value zcro (like ¢) and correlation fuuction 


rea a 5 = K&r), (9.69) 


Here, the intensity coefficient K is chosen equal to the intensity 
coefficient of the original process (9.68): 


K-= i. hd dr. (9.70) 


After we have made this replacement of ¢ by , equation (9.67) 
describes a Markov process, and is equivalent to the Fokker-Planck 
equation 

1 


= eA lrine — mye] +5 (<= 


diy 


Ci 





where 
m = aigR(etty ~ aigRe™#?? (9.72) 


[cf. (9.39)]. 
Finding the nonstationary solution of cquation (9.71) is a 
complicated problem, but the one-dimensional _ stationary 
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probability density w(x) can be found quite easily. Applying (4.49), 
we have 


w(x) == N- exp = GK |a2(In x = 5) ~ Imx' |, (9.73) 





where N is a suitable normalization constant. It is clear that the 
form of the distribution (9.73) is determined by only two parameters 


: 2 
mm = digRe*el? and 9 = {aol K. (9.74) 





The first parameter m determines the coordinate x9 with maximum 
probability, and in fact x9 is just the quantity satisfying equation 
(9.40), i.e., x9 1n x9 = m. The second parameter g determines the 
amount of ‘‘scatter” due to the fluctuations. Thus, the coordinates 
x, and x,, for which the probability density falls off to e~/? ~ 0.61 
times its maximum value, can be found as the roots of the equation 


f(x) —f(%) = 9, (9.75) 
where 
f(x) = (In x — 3) — mx. (9.76) 


From the distribution (9.73), we can easily find the probability 
density of the output voltage 7, which is related to x by the zero- 
memory nonlinear transformation 


= l Inx 
a= aine. 
Using (9.73) and formula (1.15), we obtain 
4ep Gs (de fe a1 
mon) = Hp EXP | 39 fe 2(an ~— 4) 2me| +a|. (9.77) 
Because of the presence of the term a7 in the exponential, whose 
appearance is due to the derivative dx/dj, the value 7,, corre- 
sponding to the maximum of the probability density (9.77) does not 


equal 
= lin x, 
No a 0° 
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In fact, 7, is now defined as the root of the equation 
ane" — mer —g--- 0, (9.78) 


Making a Taylor’s series expansion of the furtction which is the 
argument of the exponential in (9.77) and takes its maximum value 
at 7 = 7, we obtain 


5p (ean — B) — Ime 4 a 
= ~ 5 [eMlatm — 2) — Amer] 1 ann (9.79) 
= yy [etn 2m $1) = morn] a> — 9)? | a 
Retaining only these first few terms, we find that 


cfm) A eam) exp | ~ 2.5310) | (9.80) 
in the region of maximum probability. Flere, 


08, = Je ( arm + 1) — menrmyt 
ae (9.81) 
= ah t+ az) tem + ac} 
[cf. (9.78)] is the variance of an equivalent Gaussian distribution 
which is close to (7) in the region of maximum probability. 

If the distribution w(7) differs only slightly from a normal 
distribution, the qv.antity 7,, is apprevimately equal to the niean 
value ce and the equivalent variance o2, is approximately equal 
to the variance Dy, For distributions which are not approximately 
normal, 7,, and o2, are no longer close to (7) and Dy, respectively, 
but they still serve to characterize the distribution z(y). ‘The 
equivalent variance o%, characterizes the amount of “fluctuational 
scatter,” in the sense that Dy ~ o3,. If we want to determine the 
actual mean value and variance in the case where 2(7) is not 
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approximately normal, we can calculate the appropriate integrals 
involving the weight (9.77), resorting to numerical integration if 
necessary. 

For Gaussian noise &(t), with 


)=0, <b = oF Rr), 
we have 


K = 2earo? J * [eto Ide, (9.82) 
o 


according to (9.54) and (9.70). For some specific forms of the 
correlation coefficient R(r), we can immediately calculate this last 
integral. For example, if 





-, (9.83) 


Toor 


Ryser, oy 


i 


then, making the change of variable 


z= Borer, 


we find that 


Kate [alg =? eto! [E*(a%o?) — 2In ao — C]. (9.84) 
= 5 P re =o (a ao m 


Here, E*(z) is one of the exponential integrals,? and C = 0.577... 
is Euler’s constant. If the integral (9.82) cannot be calculated 
directly, the expansion 


K =e p {oof f : R%(r) dr (9.85) 


may turn out to be useful. 
As an example, suppose that the parameters have the values 


as & 100. 


Teor 


ao =3, aiyR = 10, yRC = 





2See Bateman Manuscript Project, op. cit., p. 143. 
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Then 
m= 898,  ¢ = 8.25 > 10°, 


and, using (9:41), (9.78) and (9.8}), we find that 
ano = 5.16, ay = 7.05, tay, = 0.655. 


On the other hand, if ao and aiy)R have the same values, while the 
capacity is increased }0 times, so that yRC == 1000, then 


q=825x 10°, an, = 6.1, aoe, == 0.543. 


By comparing these results, we see that when the capacity C is 
increased, both the mean value of the voltage across the load and its 
fluctuational scatter decrease. 

From the above quantitative results, pertaining to Gaussian noise 
with the specific correlation function (9.83) we can also conclude 
that the readings of a voltmeter which measures the mean value of 
the output voltage will be different for different valucs of 7,,,/RC, 
even if the variance o? of the input noise remains the same. There- 
fore, it must be admitted that the use made of vacuum-tube 
voltmeters in ordinary engineering practice, to measure noise 
variances, is erroneous. Vacuum-tube voltmeters can be used only 
for comparative estimates of noise intensities, and only for noise 
whose spectral composition does, not change. ‘Mhermoelectric 
devices should be used to measure actual mean square valucs of the 
noise fluctuations. 


4. Detection of a Sine Wave Plus Noise with a 
Small Correlation Time 


The method for calculating the one-dimensional distribution of 
the output of a nonlinear device with memory, presented in the 
preceding sections, is also applicable when the input contains 
a “sine wave” Ecos wot, together with the random noise é&#). 
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Then, for a detector with the exponential characteristic (9.33), the 
corresponding version of equation (9.36) is 


RCE 4+xInx = at,R exp {aE cos wt + af}. (9.86) 


We now consider two special cases: 


1. Let the input noise process &(t) be slowly varying compared to 
the rapidly oscillating signal E cos wot. We assume that the period 
T) = 2m[w, of these oscillations is much less than both the 
correlation time 7,,, of the noise and the filter time constant RC. 
Then equation (9.86) can be averaged over a period, which gives 


RC + x In x = aigRI(aE) e** . (9.87) 


This equation differs from (9.36) only by having the parameter 
ai,RI(@E) instead of aipR. Therefore, with the corresponding 
changes, everything said in Secs. 2 and 3 applies to this case. 


2. The case where the process &(t) is narrow-band is somewhat 
more distinctive. We assume that the frequency band of &() lies 
near the frequency wo of the sine wave. Then, the total input signal 
can be represented as an oscillation with resultant amplitude B(1) 
and phase y(t), ie., 


Ecos wot + &t) = Blt) cos [wet + ¥(t)] . (9.88) 


We have already encountered such signals in Chap. 7 [cf. formula 
(7.80)). If &(¢) is a Gaussian process, then the probability density of 
the amplitude B is of the form (7.88). 

Substituting (9.88) into (9.86), we can use the fact that B(t) and 
y(t) are slowly varying compared to a sine wave of frequency w9. 
This allows us to average the right-hand side of (9.86) over a 
period T, = 2m/w, by regarding B(t) and (#) as constants. The 
result is 

RCé 4 xInx = ai,RI,[aB(t)) . (9.89) 
When the time constant RC greatly exceeds the time in which the 
amplitude B changes, i.e., 
Teor <RC, 
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where 7,5, is the correlation time of the noise, we can apply 
stochastic methods involving the Fokker-Planck equation. ‘Thus, 
we regard the ‘‘fluctuational component” 


Q(t) = I(aB) — <h(2B)> (9.90) 
in (9.89) as being delta-correlated, by making the change 


(E> -> Ka(r) » (9.91) 


where 


K=f is Gl) de (9.92) 


is the intensity coefficient of the process (9.90). ft is convenient to 
write 
m = atgR(I(aB)> , (9.93) 


since then (9.89) has the same form 
RCs + x Inx = m + ai,RE (9.94) 


as in the preceding section, so that formulas (9.73) and (9.77) to 
(9.81) can be applied to the present case without any changes at all. 

The parameters (9.92) and (9.93) of the process {(f) involve the 
averages (J,(aB)> and (/,(aB)1,(aB,)>. To calculate these quantities, 
it is convenient to average the random function 


er2tt) =. exp {E cos wot + a€(1)} 


before carrying out the time average over a period. hus, if &(t) is a 
Guussian random process with mean value zero and corrclation 
function 

o®R(r) = 0°r(7) Cos wy , 
we have 


cet) = exp {aE coswyt | date}, (9.95) 
Cestter ey 


= exp {aE cos wot + aE coswft +7) + ao%fl | r(7) cos oz]. 
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Averaging the first of these expressions with respect to ¢ over a 
period, we obtain 


Wy ptt iening) 


<I{aB)> = 5? j, Cert» dt’ = et*l2 J (ak), 


Averaging the second of the equations (9.95) with respect to ¢ over 
a period is equivalent to averaging the expression (8.121) with 
respect to a random initial phase. Setting 12 = 12, = a in (8.121) 
and (8.135), we find that 


Ceariaryy = evo? > €,F2(aL) et*a*reo8et cos kwyt « (9.96) 
We now average (9.96) with respect to r. According to (8.136), this 
gives 


ett > €,1?(aE) 1,(a®o"r) . 
Deel )1,(a%o"r) 


‘Therefore, we have 


ow 


If aB) 1{aB,)> =e" Y) ela) I,{a°o"r(z)] , (9.97) 


An? 
a 


KEL, =e? (aE) {Igla°o*r(r)} -- 1} + 2e%** >) (aE) I, [a*o*r(7)) « 
7 (9.98) 


To calculate the intensity coefficient K, we substitute (9.98) into 
(9.92). It may be convenient to use a series expansion of the Bessel 
function, when evaluating the integral with respect to r. Suppose 
the correlation coefficient is of the form 


Xr) sevllr, 
Then we find it useful to introduce the new variable of integration 
z= adore", 


In the special case E == 0, according to (9.98) and (9.92), we have 


K= 2 ata? rn hey dz = Leet > u (Se)" - (9.99) 


to z it mm)? 2 
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When a’o? >> 1, it is convenient to use the asymptotic represen- 
tation 
Fz) 4, . A) Is) yn faa) 
| MD de = BEE 5 DAR emt EE |. (8.100) 
It is interesting to compare the expression (9.99) with formula 


(9.84), pertaining to the case of slowly varying input fluctu.tions. 
For the value ao = 2, the intensity coefficient (9.84) is 


52010 
7 


K 


’ 


while, for “‘quasi- harmonic” input fluctuations, (9.99) gives 
q P g! 


K= Lens $2 +059 40.11 $0.03 +. < 2. 


5. The Case of Intermediate Correlation Times 


The most difficult case is where the correlation time of the input 
process é(#) in the equation 


Ma ill 1 
#1 pon = aE -0 (9.101) 


is comparable with the filter time constant RC. ‘Then, we can use 
neither the quasi-static model, nor the method of the Fokker- 
Planck equation in the form given aboye. However, in this case, we 
can stil} use stochastic methods involving the Fokker-Tlanck 
equation, if we consider two-dimensional Markov processes. 

Thus, let &2) be a Gaussian process with mean value zero and 
correlation function 


Eby = oe rl, (9.102) 


As we know from Chap. 4, Sec. 10, such a process is Markovian 
and satisfies the differential equation 


E+ ye — U1), (9.103) 
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where {(t) is a Gaussian delta-correlated process such that 
CO -5 0, ELD = 2y0°8(r) = Ky S(r) - 


Equations (9.101) and (9.103) together specify a two-dimensional 
Markov process. Writing v= €— 7, we can replace these 
equations by the new system 


, v 1 1 

9=— Ra GA — (ry Reh EEE 

F (9.104) 
=~rvi+l, 
with the equivalent Fokker-Planck equation 

v 1 1 ow 

me.) = Sollee + CPO] | + (Re) ES 
(9.105) 


a ow 
+ gg (Em) + yo? (= ove +2 amet Se). 


Using this equation to find the probability density wv, €) or 
wv, €; v,, €,), we can then integrate with respect to € and &, to 
obtain 


wn) = [ 6 — 2 6) 46 , 
ulm, ne) = { E05 65 be — 40 be) dE de 


Unfortunately, the complete solution of equation (9.103) is a 
difficult problem, and hence we shall confine ourselves to special 
cases. 


5.1. First, we consider the case where the correlation time is 
exactly equal to the filter time constant, so that 


1 


Y= RO (9.106) 
Then the voltage across the diode satisfies the equation 
b=—- 2-2 +H), (9.107) 


RC c 
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and since it docs not contain é, this equation can be solved indepen- 
dently. From the process z(£), we can obtain the output veltage by 
using (9.101). The result is 


1 vt . { 
HO GP err} aeeedr (108) 
since 
1 Vg. 
at Rey GF. 


The average of the stationary value of » equals 
<n = RiF(e), 5 (9 109) 
and to find the output correlation function, it is sufficient to know 
(FO) Fw). 
The Fokker-Planck equation corresponding to (9.107) is 


@(v) = x Zt + RF(2)) rh | yori: (9.110) 


which has the stationary solution 


ve Rr. i 
wv) = ay ee |— gr -srf Pee. en) 


To find (F(v)), we have to evaluate the integrals 


<F(e)> xl. exp | a af. F(e)de! Fey de (9.112) 


and 








N= i exp | a - R f F(z) del de, 





If the current-voltage characteristic of the diode is piecewise linear, 
of the form 


Sv for v>0O, 
Fv) = io few. (9.113) 
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these formulas Icad to the results 
ie 225, a 
COP TERS” 
(9.114) 


N= yu RE RSY "A, 


and hence, according to (9.109), 


2 RSo 
on ae oe 





We get different expressions for (F(v)>, N and <n) if the 
characteristic F(v) has a different form. In what follows, we shall not 
specify the precise form of the function F(v), but we shall assume 
that it vanishes for v <0, and grows rapidly for v > 0, in fact, 
much more rapidly than v/R. ‘This second condition is satisfied in 
most practical cases, where the diode has a small internal resistance 
R; <R in its conducting state, so that the ratio of the output 
voltage of the detector to the input voltage is close to unity. Under 
these conditions, the probability 


To 


0 
dy 3 i} w(v) dv — in (9.116) 
that the diode is in the nonconducting state greatly exceeds the 
probability 


a 1; Re 
we [: w(v) dv ~ W I, exp | - aio) da! do (9.117) 


that it is in the conducting state. Moreover, the inequality 
WwW, KW, (9.118) 
shows that most of the time the diode is in the nonconducting state. 


In fact, it is clear that 


w, el, Waeal, (9.119) 
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To calculate the average (9.112), we can neglect the term 77/2 in the 
exponential compared to R SF) dz, obtaining 
paige (REP pare aes j2 ¢ 
<F(v)> = velf exp} aa] Me) de ey de Woes 
(9.120) 
so that 


<> = Aes, 


Turning to the calculation of the correlation function, we fix 
the initial position 7(0) = vw of the “representative point” (see 
p. 63). Suppose that v) < 0. Then, according to (9.107), we have 


d+yo = Ut), (9.121) 


as long as v is negative, since the function F(z) has no effect in the 
region where v is negative. Using (9.121), we can find the average 
displacement and the “fluctuational scatter” of the representative 
point: 








<u> = we", 
(9.122) 
Do = 0X1 — e-®), 
Therefore, v(t) is described by the-probability density 
! ! 1 (v_- ve 1)? | 
mle, 10) = om ees exp | gia METI, (0.123) 
which is the solution of the Fokker-Planck equation 
—  Avw) | 2 Pe 
bay bw SG. (9.124) 


When the representative point reaches the region where 7 > 0, 
the function F(v) comes into play, and reflects the representative 
point back into the region where v < 0. In fact, there remains 
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only a very small probability w,<1 of penetration into the 
region where v > 0. Therefore, we can assume that the probability 
current (sce p. 62) through the boundary v = 0 vanishes, ie., 





which leads to the boundary condition 
00) _ 9, (9.125) 


This condition is not satisfied by the function (9.123); the solution 
of (9.124) which does satisfy (9.125) has the form 


i] 1 
wv, t | %) = —— S==— 
MON (9.126) 
.(v — ae-")?) Lb (v + uge-74)? 
x [exp | — 202 1 oe + exp} - ptt T— ae ‘] 





(v<0,% <0). 


Then at the origin we have 





1 ode?r! N 
WO, t | %) = a ES Wm oP — 3 T ea (9.127) 


and the function is extended in a continuous fashion into the 
region v > 0. In this region, the distribution is practically the same 
as the stationary distribution 


pe tvt 


wv, t |v) = —=— exp aH 
byt zt 2ot | — e-2vt 
Tis me. Wa aor . @ tenes (9.128) 


& f ; F(z) ds| 


vw 
x exp Fae 





(2 >0,% <0), 


which has the same value at the origin as (9.126). 
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If the initial point approaches the houndars, ic. 7 ~0, the 
function (9.126), (9.128) goes into 


2 4 a ee ee a ae 
wv, t | 0) wovl tem Py “Beth oe 4 fore 0, 
v, = g 
2 (eRe 
Niovi wet PN” BF oot { Flevde(tore 0, 
(9.129) 


When the initial valuc vy lies in the region where 7 is positive 
(so that v > 0), then, due to the influence of F(z), the represen- 
tative point is rapidly ‘pushed out” into the region where & is 
negative. After this occurs, the distribution ar | 7) is close to the 
distribution (9.129) just found. In fact, the two distributions differ 
only during a time of order (yRF’)"! =2 R,C, which is negligibly 
small compared to I/y = RC, because ok the assumption nade 
above. 

Once we know the one-dimensional distribution (9.111) and the 
transition probability z(v, ¢| v9), we can write down the two- 
dimensional probability density 


wD, V,) = w(v) (v,, 7 |v) 


and the second moment 

<F(e) Flo)> = [J FO) Fle) wv) een 717) ded, (9.130) 

0 0 
Taking w(2,, 7 | v) in (9.130) to be (9.129), we find that 
1 tS 
FFD = WV ae 
x [f- Fe) ex \- as ~3 pK) dz! de i (9.131) 
- P Qo® eI, PS ey 


2 <F(e)® = 
= wap? oe er eo 
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Because of (9.119) and (9.120), this gives 


2 a 


kyr) =~ CF(v) Fle) — Fay 22 ~ — i]. (9.132) 


mlyines Vine 
Vor small 7 < R,C, the expression (9.132) is not valid, and for 
7 = 0, we cat use the one-dimensional distribution (9.111), which 
leads to 


h,(0) = DF(w) ae I. exp} - _ [Fe dz! FY) do. (9.133) 


‘Vo find the spectral density S[F(v) — <F); w] for frequencies w 
of order y and smaller, we can take the Fouricr transform of (9.132). 
The region of smal! valucs of 7, where (9.132) is not valid, will 
have an important effect only for high frequencies w ~ 1/R;C. The 
Fourier transform of (9.131) can be found by using the obvious 
relation 


lel bere a ~ Hf dr = ae fe [A tae 


where p = 7w, and the formula 


is dr I dt 1 p t 
pr wt /2 =——B = 
ie VI - err =a, wl : Gea). 
(9.134) 





where A(x, v) is the beta function. Comparing the last two 
equations, we sce that 


var awl 1) _ 2y 
. La -- e-vitl t] a 1 Re [2 (E. 3) tw)” 
and hence, according to (2.11) and (9.132), 


iwRC 
40°C (i 4): 


STR) Fy ia) = 2 Re B (9.135) 


2 See I. M. Ryshik and 1. S. Gradstein, op. cit., formula (5.218.3), p. 258. 
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To determine the spectral density numerically, we can use 
tables of the gamma function, recalling that 


* Biz, 4) = V0 re iy 


or else we can use the expansion 


 (2k-- 1)! I 
Bi ) = Dom ge — i is Th 








To improve the convergence of the last formula, we transform it 
into! 
&  (- I! i 
ot 2-# Da iste sa alt 
B (iz, 3) at 2In a2. 1 P-igi(k ijl Wie 1? 


which implies that 


Re B (iz, }) =2In2 — 2? > aie Oo Me Were ay 0-136) 





Substituting (9.136) into (9.135), we have 


SF) --<F) 54 =4 a aD In2 





“bie 


1-3-5 
+296 soa tll: 


* Use the fact that 


— (2k — 1)! 
2m? = Der RDE 


which follows from 


po A(z i 
2In2 =f Ws St as 
o 


if the integrand is expanded in a power series and integrated term by term. 
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where 2 == w/2y. This function, which gives the spectral density of 
the fluctuations of the current J = F(v), is plotted in Figure 15. 





Fig. 15. The spectral density of the fluctuations of the current | = F(v). 


According to (9.101), the output voltage 7 is obtained from F(z) 
by a linear transformation. It follows from the usual rules and 
(9.135) that the spectral density of the fluctuations of the output 
voltage is 


Sly — ( ;0] = | iwC 4 _ \" S[F(v) — CF) 5 0] 
(9.137) 





Finally, according to (2.15), to find the variance Dy, we have to 
integrate (9.137) with respect to the frequency®: 





Dy = otf Rea (= ae = (1—2) o# © 0.363 0%. (9.138) 








° To evaluate the integral in (9.138), we use (9.136) and the easily verified 
formula 





i stds _7_t 
o G2 +1)? 4+ 4k) 22k FI" 
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5.2, When the noise correlation time |v is differcnt from RC, we 
have to solve the two-dimensionel Fokker-Planck eqpraton (9.105). 
This can be done by using methods similar to those given in 
Chap. 4, Sec. tt. Thus, to find the stationary distribution satis- 
fying the equation 





Cee v L.,. yy re 
ve | Filles | eM) de 
; (9.139) 
+7 ag (Ee) 1&2. 1 Dy 0, 
where 
1 
m7 RE 
we look for a solution of the form 
22) &) = DY Tuan Vinlt) XE) (9.140) 
myn-0 
Here, the functions 
X,(8) = rovn (2) (9.141) 
are the eigenfunctions of oo (4.68), which satisfy the relations 
» Vn¥1y. 
eS gen (9.142) 


eX, = —olVnXy tV0 dV Xv] 





obtaining 


2 ld Qk 1)! ! 
J, rea(, oe *[in2 Lee TF OR | 





According to I. M. Ryshik and I. S. Gradstein, op. cit., formulas (1.6411) 
and (1.642.2), p. 48, 





> (2k — 1)! xin 
2-TRNR — I)! 2k(2k + 1) 


2 | f 
xin ( — are cosh -) arcsiny [ox 
ro 2 x 


for x < I. If we set x — 1, (9.138) now follows from the last two formulas. 
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The eigenfunctions V,,(v) satisfy the equation 





+A (le 1 ri) Vy) bayln = 0. (9.143) 


Substituting (9.140) into (9.139), and taking into account (9.142) 
and (9.143), we obtain 


Phat eX 5 Dahavar, 
ioe mn 
va (9.148 
ahd > TyeVinl V0 Xp + V0 +1 Xa 


myn 


yo D) Tbr Va-F 1 Xn = 0. 


ann 


Next, we expand the derivatives V, with respect to the cigen- 
functions V,,, by writing 


epee 7 
Vis > Fans 
m 


my 


where 


m 


-, dv 
a -fvue. 9.145 
mk f A Ve ( ) 


Then, from (9.144) we obtain the equations 


On 48) Tan = 2 — Qe Vn D ane Free “a 
6 
— (9.146) 
~erVn $1 DameT ens , 
F 


which can be solved by various methods. Having specified a given 
accuracy, we need only consider a certain number of expansion 
coefficients 7°,,,, of low order, setting the higher-order coefficients 
equal to zero. ‘Then, (9.146) gives a finite system of linear equations, 
from which we can determine the 77,,,,. Another way of solving the 
problem is to use the method of successive approximations, 
regarding « as a small parameter, and obtaining the solution 
in the form of an expansion in powers of this parameter. 
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x Comhining the expressions obtained from (9.146) for » = 0, 
n= Landn = 2, we find that 


Tmo = — €(2y — €) 0? > but Tro 
rs 


(9.147) 
2A, 
+ %{2y — 6) o' ot Debi a but 1 OF) (nm +0), 
where we have written 
1 ay, a 
bee Gomon | 
TN SOM Ey” 


By repeatedly substituting the whole right-hand side of (9.147) for 
Tro, T 19, --, We can express the cocfficients T,,9 in terms of Toy: 


Tyg = — (2y ~ €) Oyo eg 
(9.148) 
4 EQy oot Dwg aoe bilo 1 Ole) (m= 0). 
h#0 + 


To determine the other coefficients T,,,,, we again turp to the 
equations (9,146), applying the method of successive approxima- 
tions. Letting 7/9) denote the zeroth approximation, i.c., 


Poa Tin | O(), 


we find from (9.146) that 


Ty = Qy — a 5 7 Oe 


vi 
Dy — Srna el a 
Tad = Oy — 055s Beta 9) 


Bur 


ns 2S g\22 y 
= Qy — «Po 5 wg EH Tos 
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The zeroth approximation is then used to find the first approxima- 
tion to T;,"., which differs from T,,, by terms of order e: 


a 10) 
2 mil ee ’ 


not y 





Pa) PO) (2y —- «)o 
Ta = Te + tek y tnt Te ey 


v2 v3 
TW = (2y— eo i DauTit --¢ at ape Dama T iy 5 
(9.150) 


This method allows us to find T,,, to within terms of order ¢*, 
where p is any integer. Naturally, the larger», the more formidable 
the calculations become.* 

All the coefficients T,,,, in the expansion (9.140) turn out to be 
proportional to the first coefficient To9, which is determined by the 
normalization coefficient. In fact, if we integrate (9.140) with 
respect to v and &, all the terms except the first vanish, because of 
(4.54), and hence 


J w(v, £) dudé =Ty = 1. (9.151) 


To simplify the calculations, it is convenient to assume (as on 
pp-274-277) that the function F(v) vanishes for v <0 and grows 
rapidly for v > 0. More precisely, tet R, = (AF /év) -1, the internat 
resistance of the diode in its conducting state, satisfy the conditions 


R,<R, RC<!. (9.152) 
Then, in considering the region v > 0, we can neglect the terms 


i 
pe W-Reé 


in the first of the equations (9.104), in comparison with F(v)/C; 
instead, we can use the equation 


jes LFO) dots (9.153) 
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The corresponding Fokker-Planck equation has the stationary 
solution 
= 1 aeey aa! 
tw(v) — const - exp} ~ Fee fre) ds, (9.154) 

which is “established” after a time exceeding the small time 
constant R,C. 

For the region v <0, we can replace the function /(7) in 
equations (9.104) and (9.105), by the function 


Fu) = |e for 050. (9155) 
The infinite value of F,(v) for v > 0 means that the representative 
point is immediately reflected back when it reaches the boundary 
v = 0. Actually, the reflection takes a time of order R,€', which is 
considerably less than the time constants 1/RC and { y which 
characterize the process for v < 0. 

After replacing F(v) by (9.155), equation (9.143) has the eigen- 
functions 


2 1 


Vv) V (2m)! o VyRC 








Fens ( (7 <0) (9.156) 


ae) 
o VyRC 
[see formula (4.70)], and the eigenvalues 


2m ° 


an = RE 


According to (9.145), in the present case, using the formula’ 


J Fan(2) Hoxss(2) 77? dz 
° 


__ Qm— 1) (2k — I 
DIM - WE RI(2m 2k it)’ 





= ( lyn 


* Cf. Bateman Manuscript Project, op. crt., formula (9), p. 117; formula (21), 
p. 122; formula (7), p. 3. 
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we have 


(cS) a aL (2m - I) (2k + I)! 





Anh = 


Gg VamyRC V (2m)! (2A)! 2" im — DERI Qm —- 2k — 1)" 
(9.157) 


If we use (9.146) to calculate 7',,,, to some desired accuracy, then, 
according to (9.140), we obtain the jaint stationary probability 
density of the input and output signals, Le., 


wal £) = (OCD, Yemen = Dy Tuna ~ 9) Xa), (9-158) 


where the functions V,,, and X, are given by formulas (9.156) and 
(9.141). By using expansions of this kind, we can also investigate 
nonstationary transient processes, and two-dimensional probability 
densities corresponding to different instants of time. 
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